INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 37, size: 170,637,559. OK num: 37(download 37 objects).

average speed 110230000(byte/s)

1.551720(s) elapsed
INFO: Downloading succeed.
Network is under initialization...
Network successfully initialized.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.49139968, 0.48215827, 0.44653124)
	std: (0.24703233, 0.24348505, 0.26158768)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.41M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.2
search_space = s1
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-10-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-10-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 2.498 (2.50)  Acc@1:  0.0000 ( 0.0000)  Acc@5: 32.8125 (32.8125)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 1.641 (1.98)  Acc@1: 35.9375 (25.3676)  Acc@5: 93.7500 (78.3701)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 1.874 (1.85)  Acc@1: 35.9375 (30.6002)  Acc@5: 85.9375 (82.9827)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 1.500 (1.76)  Acc@1: 54.6875 (34.4578)  Acc@5: 89.0625 (85.3063)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 1.817 (1.70)  Acc@1: 35.9375 (37.2201)  Acc@5: 93.7500 (86.9325)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 1.267 (1.64)  Acc@1: 56.2500 (39.3240)  Acc@5: 93.7500 (87.9793)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 1.272 (1.60)  Acc@1: 60.9375 (41.1337)  Acc@5: 85.9375 (88.7458)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 1.292 (1.56)  Acc@1: 51.5625 (42.7929)  Acc@5: 98.4375 (89.5121)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 1.168 (1.53)  Acc@1: 50.0000 (43.9960)  Acc@5: 95.0000 (90.0280)LR: 2.500e-02
train_acc 43.996000
Valid: 0 [   0/390]  Loss: 1.124 (1.12)  Acc@1: 60.9375 (60.9375)  Acc@5: 93.7500 (93.7500)
Valid: 0 [  50/390]  Loss: 1.317 (1.22)  Acc@1: 57.8125 (55.3309)  Acc@5: 96.8750 (95.6495)
Valid: 0 [ 100/390]  Loss: 1.218 (1.22)  Acc@1: 56.2500 (55.4301)  Acc@5: 93.7500 (95.3899)
Valid: 0 [ 150/390]  Loss: 1.091 (1.22)  Acc@1: 64.0625 (55.6705)  Acc@5: 98.4375 (95.2297)
Valid: 0 [ 200/390]  Loss: 1.510 (1.22)  Acc@1: 48.4375 (55.7369)  Acc@5: 93.7500 (95.2425)
Valid: 0 [ 250/390]  Loss: 1.296 (1.22)  Acc@1: 57.8125 (55.7707)  Acc@5: 92.1875 (95.0697)
Valid: 0 [ 300/390]  Loss: 1.264 (1.23)  Acc@1: 43.7500 (55.8399)  Acc@5: 96.8750 (95.0374)
Valid: 0 [ 350/390]  Loss: 0.9657 (1.23)  Acc@1: 68.7500 (55.9072)  Acc@5: 100.0000 (94.9653)
Valid: 0 [ 390/390]  Loss: 1.356 (1.23)  Acc@1: 52.5000 (55.8480)  Acc@5: 87.5000 (94.9600)
valid_acc 55.848000
epoch = 0   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4961, 0.5039],
        [0.4912, 0.5088],
        [0.4944, 0.5056],
        [0.4862, 0.5138],
        [0.4913, 0.5087],
        [0.5175, 0.4825],
        [0.4873, 0.5127],
        [0.4804, 0.5196],
        [0.4827, 0.5173],
        [0.4968, 0.5032],
        [0.5133, 0.4867],
        [0.4901, 0.5099],
        [0.5069, 0.4931],
        [0.4946, 0.5054]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5276, 0.4724],
        [0.5108, 0.4892],
        [0.5285, 0.4715],
        [0.5260, 0.4740],
        [0.4975, 0.5025],
        [0.5285, 0.4715],
        [0.5128, 0.4872],
        [0.4985, 0.5015],
        [0.4980, 0.5020],
        [0.5285, 0.4715],
        [0.5275, 0.4725],
        [0.4916, 0.5084],
        [0.4940, 0.5060],
        [0.4907, 0.5093]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 1.379 (1.38)  Acc@1: 50.0000 (50.0000)  Acc@5: 89.0625 (89.0625)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 1.130 (1.20)  Acc@1: 60.9375 (57.0159)  Acc@5: 96.8750 (94.7917)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 1.030 (1.17)  Acc@1: 60.9375 (57.7042)  Acc@5: 96.8750 (95.4208)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 1.359 (1.16)  Acc@1: 56.2500 (58.2368)  Acc@5: 92.1875 (95.3746)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 0.9375 (1.15)  Acc@1: 62.5000 (58.6521)  Acc@5: 96.8750 (95.4991)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 0.9705 (1.14)  Acc@1: 60.9375 (59.0575)  Acc@5: 98.4375 (95.6424)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 0.9903 (1.13)  Acc@1: 65.6250 (59.4632)  Acc@5: 98.4375 (95.7122)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 1.045 (1.12)  Acc@1: 64.0625 (60.0027)  Acc@5: 96.8750 (95.8022)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 0.8760 (1.11)  Acc@1: 72.5000 (60.3560)  Acc@5: 97.5000 (95.8360)LR: 2.498e-02
train_acc 60.356000
Valid: 1 [   0/390]  Loss: 1.257 (1.26)  Acc@1: 53.1250 (53.1250)  Acc@5: 96.8750 (96.8750)
Valid: 1 [  50/390]  Loss: 1.052 (1.15)  Acc@1: 64.0625 (59.3444)  Acc@5: 96.8750 (96.5074)
Valid: 1 [ 100/390]  Loss: 0.9246 (1.18)  Acc@1: 65.6250 (58.3849)  Acc@5: 98.4375 (96.2407)
Valid: 1 [ 150/390]  Loss: 1.375 (1.21)  Acc@1: 57.8125 (57.6987)  Acc@5: 92.1875 (95.9437)
Valid: 1 [ 200/390]  Loss: 0.9447 (1.20)  Acc@1: 67.1875 (57.6959)  Acc@5: 98.4375 (95.9888)
Valid: 1 [ 250/390]  Loss: 1.026 (1.20)  Acc@1: 65.6250 (57.8499)  Acc@5: 98.4375 (96.1467)
Valid: 1 [ 300/390]  Loss: 1.186 (1.19)  Acc@1: 64.0625 (58.0980)  Acc@5: 93.7500 (96.1534)
Valid: 1 [ 350/390]  Loss: 1.055 (1.20)  Acc@1: 62.5000 (57.8570)  Acc@5: 98.4375 (96.0515)
Valid: 1 [ 390/390]  Loss: 1.416 (1.20)  Acc@1: 55.0000 (57.8280)  Acc@5: 92.5000 (96.1280)
valid_acc 57.828000
epoch = 1   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 2), ('dil_conv_3x3', 3), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4900, 0.5100],
        [0.4814, 0.5186],
        [0.4900, 0.5100],
        [0.4715, 0.5285],
        [0.4806, 0.5194],
        [0.5280, 0.4720],
        [0.4813, 0.5187],
        [0.4635, 0.5365],
        [0.4717, 0.5283],
        [0.4909, 0.5091],
        [0.5223, 0.4777],
        [0.4892, 0.5108],
        [0.5134, 0.4866],
        [0.4897, 0.5103]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5447, 0.4553],
        [0.5155, 0.4845],
        [0.5429, 0.4571],
        [0.5407, 0.4593],
        [0.4921, 0.5079],
        [0.5447, 0.4553],
        [0.5176, 0.4824],
        [0.4916, 0.5084],
        [0.4883, 0.5117],
        [0.5445, 0.4555],
        [0.5423, 0.4577],
        [0.4796, 0.5204],
        [0.4790, 0.5210],
        [0.4775, 0.5225]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 0.8692 (0.869)  Acc@1: 67.1875 (67.1875)  Acc@5: 98.4375 (98.4375)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 1.077 (0.990)  Acc@1: 60.9375 (64.3689)  Acc@5: 98.4375 (96.6912)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 0.8603 (0.987)  Acc@1: 70.3125 (64.7432)  Acc@5: 100.0000 (96.7358)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 0.7415 (0.967)  Acc@1: 78.1250 (65.3767)  Acc@5: 98.4375 (96.8336)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 1.024 (0.954)  Acc@1: 64.0625 (66.0370)  Acc@5: 95.3125 (96.9139)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 1.035 (0.950)  Acc@1: 68.7500 (66.2226)  Acc@5: 96.8750 (96.8937)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 0.7740 (0.941)  Acc@1: 75.0000 (66.5386)  Acc@5: 96.8750 (97.0255)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 0.7891 (0.933)  Acc@1: 70.3125 (66.7824)  Acc@5: 95.3125 (97.1287)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 0.7857 (0.926)  Acc@1: 72.5000 (67.1600)  Acc@5: 100.0000 (97.1880)LR: 2.491e-02
train_acc 67.160000
Valid: 2 [   0/390]  Loss: 0.9081 (0.908)  Acc@1: 60.9375 (60.9375)  Acc@5: 98.4375 (98.4375)
Valid: 2 [  50/390]  Loss: 0.9041 (0.944)  Acc@1: 68.7500 (66.5748)  Acc@5: 93.7500 (97.2120)
Valid: 2 [ 100/390]  Loss: 0.8800 (0.976)  Acc@1: 70.3125 (65.3620)  Acc@5: 98.4375 (97.1071)
Valid: 2 [ 150/390]  Loss: 0.7819 (0.966)  Acc@1: 64.0625 (65.7492)  Acc@5: 98.4375 (97.3303)
Valid: 2 [ 200/390]  Loss: 0.9382 (0.963)  Acc@1: 64.0625 (65.6328)  Acc@5: 98.4375 (97.2948)
Valid: 2 [ 250/390]  Loss: 0.9031 (0.956)  Acc@1: 68.7500 (65.8055)  Acc@5: 98.4375 (97.3917)
Valid: 2 [ 300/390]  Loss: 0.9347 (0.954)  Acc@1: 64.0625 (65.8690)  Acc@5: 96.8750 (97.3578)
Valid: 2 [ 350/390]  Loss: 0.9941 (0.950)  Acc@1: 67.1875 (66.0434)  Acc@5: 96.8750 (97.3380)
Valid: 2 [ 390/390]  Loss: 0.7698 (0.949)  Acc@1: 77.5000 (66.0520)  Acc@5: 97.5000 (97.3880)
valid_acc 66.052000
epoch = 2   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4826, 0.5174],
        [0.4814, 0.5186],
        [0.4831, 0.5169],
        [0.4684, 0.5316],
        [0.4744, 0.5256],
        [0.5381, 0.4619],
        [0.4743, 0.5257],
        [0.4610, 0.5390],
        [0.4601, 0.5399],
        [0.4875, 0.5125],
        [0.5304, 0.4696],
        [0.4837, 0.5163],
        [0.5198, 0.4802],
        [0.4923, 0.5077]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5555, 0.4445],
        [0.5186, 0.4814],
        [0.5517, 0.4483],
        [0.5463, 0.4537],
        [0.4847, 0.5153],
        [0.5539, 0.4461],
        [0.5182, 0.4818],
        [0.4871, 0.5129],
        [0.4839, 0.5161],
        [0.5535, 0.4465],
        [0.5478, 0.4522],
        [0.4713, 0.5287],
        [0.4728, 0.5272],
        [0.4709, 0.5291]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 0.8682 (0.868)  Acc@1: 70.3125 (70.3125)  Acc@5: 96.8750 (96.8750)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 0.8091 (0.829)  Acc@1: 70.3125 (71.2316)  Acc@5: 98.4375 (97.5184)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 0.7054 (0.845)  Acc@1: 78.1250 (70.4981)  Acc@5: 96.8750 (97.5248)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 0.6152 (0.836)  Acc@1: 81.2500 (70.6747)  Acc@5: 100.0000 (97.6718)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 1.057 (0.834)  Acc@1: 60.9375 (70.9188)  Acc@5: 100.0000 (97.7223)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 0.5609 (0.827)  Acc@1: 78.1250 (70.9163)  Acc@5: 100.0000 (97.7714)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 0.7396 (0.821)  Acc@1: 70.3125 (71.0963)  Acc@5: 98.4375 (97.8613)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 0.8546 (0.817)  Acc@1: 75.0000 (71.2785)  Acc@5: 93.7500 (97.8454)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 0.8878 (0.815)  Acc@1: 70.0000 (71.3800)  Acc@5: 100.0000 (97.8640)LR: 2.479e-02
train_acc 71.380000
Valid: 3 [   0/390]  Loss: 0.7583 (0.758)  Acc@1: 75.0000 (75.0000)  Acc@5: 96.8750 (96.8750)
Valid: 3 [  50/390]  Loss: 0.7397 (0.818)  Acc@1: 78.1250 (70.3125)  Acc@5: 98.4375 (97.9167)
Valid: 3 [ 100/390]  Loss: 0.8004 (0.824)  Acc@1: 70.3125 (70.1733)  Acc@5: 96.8750 (97.7259)
Valid: 3 [ 150/390]  Loss: 0.7462 (0.823)  Acc@1: 79.6875 (70.6022)  Acc@5: 98.4375 (97.7339)
Valid: 3 [ 200/390]  Loss: 0.8674 (0.820)  Acc@1: 64.0625 (70.7323)  Acc@5: 95.3125 (97.8389)
Valid: 3 [ 250/390]  Loss: 0.8533 (0.819)  Acc@1: 70.3125 (70.9226)  Acc@5: 98.4375 (97.8399)
Valid: 3 [ 300/390]  Loss: 0.8333 (0.818)  Acc@1: 75.0000 (70.8316)  Acc@5: 96.8750 (97.8821)
Valid: 3 [ 350/390]  Loss: 0.9774 (0.815)  Acc@1: 68.7500 (70.9891)  Acc@5: 98.4375 (97.8989)
Valid: 3 [ 390/390]  Loss: 0.8332 (0.817)  Acc@1: 57.5000 (71.0840)  Acc@5: 97.5000 (97.8520)
valid_acc 71.084000
epoch = 3   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4775, 0.5225],
        [0.4835, 0.5165],
        [0.4787, 0.5213],
        [0.4618, 0.5382],
        [0.4674, 0.5326],
        [0.5369, 0.4631],
        [0.4609, 0.5391],
        [0.4532, 0.5468],
        [0.4482, 0.5518],
        [0.4908, 0.5092],
        [0.5341, 0.4659],
        [0.4847, 0.5153],
        [0.5247, 0.4753],
        [0.4968, 0.5032]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5621, 0.4379],
        [0.5178, 0.4822],
        [0.5573, 0.4427],
        [0.5515, 0.4485],
        [0.4799, 0.5201],
        [0.5577, 0.4423],
        [0.5155, 0.4845],
        [0.4873, 0.5127],
        [0.4875, 0.5125],
        [0.5599, 0.4401],
        [0.5541, 0.4459],
        [0.4724, 0.5276],
        [0.4695, 0.5305],
        [0.4722, 0.5278]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 0.8655 (0.865)  Acc@1: 68.7500 (68.7500)  Acc@5: 95.3125 (95.3125)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 0.7731 (0.742)  Acc@1: 73.4375 (74.3566)  Acc@5: 98.4375 (98.1005)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 0.8952 (0.745)  Acc@1: 70.3125 (74.1491)  Acc@5: 95.3125 (98.1281)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 0.6957 (0.759)  Acc@1: 68.7500 (73.4685)  Acc@5: 100.0000 (98.0753)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 0.6587 (0.759)  Acc@1: 78.1250 (73.3287)  Acc@5: 96.8750 (98.1266)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 0.8697 (0.752)  Acc@1: 68.7500 (73.5807)  Acc@5: 100.0000 (98.1449)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 0.7702 (0.746)  Acc@1: 73.4375 (73.8216)  Acc@5: 98.4375 (98.2195)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 0.7634 (0.742)  Acc@1: 76.5625 (73.9183)  Acc@5: 100.0000 (98.2461)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 0.4422 (0.740)  Acc@1: 87.5000 (74.0720)  Acc@5: 100.0000 (98.2760)LR: 2.462e-02
train_acc 74.072000
Valid: 4 [   0/390]  Loss: 0.8206 (0.821)  Acc@1: 70.3125 (70.3125)  Acc@5: 98.4375 (98.4375)
Valid: 4 [  50/390]  Loss: 1.105 (0.775)  Acc@1: 57.8125 (73.0392)  Acc@5: 98.4375 (98.4988)
Valid: 4 [ 100/390]  Loss: 0.6982 (0.785)  Acc@1: 75.0000 (72.8187)  Acc@5: 96.8750 (98.3137)
Valid: 4 [ 150/390]  Loss: 0.5780 (0.776)  Acc@1: 78.1250 (72.6511)  Acc@5: 100.0000 (98.3030)
Valid: 4 [ 200/390]  Loss: 0.7056 (0.777)  Acc@1: 76.5625 (72.5202)  Acc@5: 98.4375 (98.2743)
Valid: 4 [ 250/390]  Loss: 0.8482 (0.785)  Acc@1: 75.0000 (72.3855)  Acc@5: 98.4375 (98.1885)
Valid: 4 [ 300/390]  Loss: 0.5506 (0.785)  Acc@1: 82.8125 (72.5862)  Acc@5: 98.4375 (98.1572)
Valid: 4 [ 350/390]  Loss: 0.7193 (0.786)  Acc@1: 75.0000 (72.7030)  Acc@5: 98.4375 (98.1348)
Valid: 4 [ 390/390]  Loss: 0.8515 (0.785)  Acc@1: 70.0000 (72.7560)  Acc@5: 95.0000 (98.0920)
valid_acc 72.756000
epoch = 4   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4749, 0.5251],
        [0.4789, 0.5211],
        [0.4741, 0.5259],
        [0.4496, 0.5504],
        [0.4640, 0.5360],
        [0.5351, 0.4649],
        [0.4562, 0.5438],
        [0.4511, 0.5489],
        [0.4389, 0.5611],
        [0.4853, 0.5147],
        [0.5357, 0.4643],
        [0.4812, 0.5188],
        [0.5282, 0.4718],
        [0.4970, 0.5030]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5720, 0.4280],
        [0.5254, 0.4746],
        [0.5656, 0.4344],
        [0.5627, 0.4373],
        [0.4750, 0.5250],
        [0.5654, 0.4346],
        [0.5135, 0.4865],
        [0.4826, 0.5174],
        [0.4805, 0.5195],
        [0.5651, 0.4349],
        [0.5638, 0.4362],
        [0.4649, 0.5351],
        [0.4642, 0.5358],
        [0.4692, 0.5308]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 0.7484 (0.748)  Acc@1: 76.5625 (76.5625)  Acc@5: 100.0000 (100.0000)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 0.7071 (0.671)  Acc@1: 67.1875 (76.0110)  Acc@5: 98.4375 (98.5600)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 0.7219 (0.684)  Acc@1: 75.0000 (75.9437)  Acc@5: 98.4375 (98.5613)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 0.6691 (0.688)  Acc@1: 76.5625 (75.7864)  Acc@5: 98.4375 (98.5099)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 0.4612 (0.681)  Acc@1: 84.3750 (76.1350)  Acc@5: 100.0000 (98.5619)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 0.7148 (0.684)  Acc@1: 73.4375 (75.9898)  Acc@5: 100.0000 (98.5620)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 0.7585 (0.683)  Acc@1: 73.4375 (76.0382)  Acc@5: 98.4375 (98.5673)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 0.7715 (0.685)  Acc@1: 68.7500 (75.9304)  Acc@5: 98.4375 (98.5443)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 0.4309 (0.684)  Acc@1: 85.0000 (76.0240)  Acc@5: 100.0000 (98.5680)LR: 2.441e-02
train_acc 76.024000
Valid: 5 [   0/390]  Loss: 0.7271 (0.727)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 5 [  50/390]  Loss: 0.6303 (0.748)  Acc@1: 78.1250 (73.9583)  Acc@5: 100.0000 (98.4375)
Valid: 5 [ 100/390]  Loss: 0.7194 (0.751)  Acc@1: 73.4375 (73.8552)  Acc@5: 100.0000 (98.4066)
Valid: 5 [ 150/390]  Loss: 0.7599 (0.754)  Acc@1: 76.5625 (74.1618)  Acc@5: 98.4375 (98.3651)
Valid: 5 [ 200/390]  Loss: 0.8010 (0.763)  Acc@1: 73.4375 (73.8806)  Acc@5: 98.4375 (98.3442)
Valid: 5 [ 250/390]  Loss: 0.6149 (0.764)  Acc@1: 78.1250 (73.8795)  Acc@5: 100.0000 (98.3628)
Valid: 5 [ 300/390]  Loss: 1.004 (0.765)  Acc@1: 67.1875 (73.9618)  Acc@5: 95.3125 (98.2610)
Valid: 5 [ 350/390]  Loss: 0.7099 (0.762)  Acc@1: 65.6250 (73.7758)  Acc@5: 98.4375 (98.2683)
Valid: 5 [ 390/390]  Loss: 0.7604 (0.763)  Acc@1: 70.0000 (73.7400)  Acc@5: 100.0000 (98.2600)
valid_acc 73.740000
epoch = 5   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4704, 0.5296],
        [0.4783, 0.5217],
        [0.4669, 0.5331],
        [0.4357, 0.5643],
        [0.4611, 0.5389],
        [0.5327, 0.4673],
        [0.4438, 0.5562],
        [0.4447, 0.5553],
        [0.4271, 0.5729],
        [0.4846, 0.5154],
        [0.5390, 0.4610],
        [0.4783, 0.5217],
        [0.5285, 0.4715],
        [0.4990, 0.5010]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5814, 0.4186],
        [0.5285, 0.4715],
        [0.5740, 0.4260],
        [0.5715, 0.4285],
        [0.4682, 0.5318],
        [0.5727, 0.4273],
        [0.5079, 0.4921],
        [0.4812, 0.5188],
        [0.4763, 0.5237],
        [0.5724, 0.4276],
        [0.5720, 0.4280],
        [0.4632, 0.5368],
        [0.4586, 0.5414],
        [0.4703, 0.5297]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 0.4684 (0.468)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 0.6927 (0.650)  Acc@1: 70.3125 (77.3591)  Acc@5: 96.8750 (99.1422)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 0.4537 (0.632)  Acc@1: 82.8125 (77.9703)  Acc@5: 100.0000 (98.9944)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 0.6301 (0.634)  Acc@1: 76.5625 (78.0733)  Acc@5: 98.4375 (98.8514)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 0.5841 (0.629)  Acc@1: 84.3750 (78.0861)  Acc@5: 98.4375 (98.8961)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 0.5144 (0.628)  Acc@1: 82.8125 (78.1375)  Acc@5: 100.0000 (98.8670)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 0.4990 (0.634)  Acc@1: 79.6875 (78.0523)  Acc@5: 100.0000 (98.8164)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 0.7410 (0.630)  Acc@1: 76.5625 (78.2185)  Acc@5: 100.0000 (98.8114)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 0.6987 (0.631)  Acc@1: 72.5000 (78.1920)  Acc@5: 100.0000 (98.7760)LR: 2.416e-02
train_acc 78.192000
Valid: 6 [   0/390]  Loss: 0.8052 (0.805)  Acc@1: 76.5625 (76.5625)  Acc@5: 96.8750 (96.8750)
Valid: 6 [  50/390]  Loss: 0.6279 (0.696)  Acc@1: 79.6875 (75.5821)  Acc@5: 98.4375 (98.2230)
Valid: 6 [ 100/390]  Loss: 0.5597 (0.695)  Acc@1: 84.3750 (75.8354)  Acc@5: 98.4375 (98.4066)
Valid: 6 [ 150/390]  Loss: 0.9182 (0.697)  Acc@1: 71.8750 (75.9830)  Acc@5: 96.8750 (98.3961)
Valid: 6 [ 200/390]  Loss: 0.5669 (0.698)  Acc@1: 75.0000 (75.7229)  Acc@5: 100.0000 (98.4220)
Valid: 6 [ 250/390]  Loss: 0.7147 (0.696)  Acc@1: 75.0000 (75.7595)  Acc@5: 98.4375 (98.4562)
Valid: 6 [ 300/390]  Loss: 0.6929 (0.698)  Acc@1: 82.8125 (75.7267)  Acc@5: 96.8750 (98.4271)
Valid: 6 [ 350/390]  Loss: 0.4566 (0.696)  Acc@1: 87.5000 (75.8324)  Acc@5: 98.4375 (98.4019)
Valid: 6 [ 390/390]  Loss: 0.5527 (0.698)  Acc@1: 80.0000 (75.7600)  Acc@5: 97.5000 (98.3960)
valid_acc 75.760000
epoch = 6   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('max_pool_3x3', 1), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4678, 0.5322],
        [0.4739, 0.5261],
        [0.4551, 0.5449],
        [0.4271, 0.5729],
        [0.4539, 0.5461],
        [0.5319, 0.4681],
        [0.4340, 0.5660],
        [0.4452, 0.5548],
        [0.4216, 0.5784],
        [0.4813, 0.5187],
        [0.5393, 0.4607],
        [0.4751, 0.5249],
        [0.5248, 0.4752],
        [0.4965, 0.5035]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5964, 0.4036],
        [0.5336, 0.4664],
        [0.5868, 0.4132],
        [0.5818, 0.4182],
        [0.4625, 0.5375],
        [0.5847, 0.4153],
        [0.5087, 0.4913],
        [0.4781, 0.5219],
        [0.4754, 0.5246],
        [0.5855, 0.4145],
        [0.5813, 0.4187],
        [0.4629, 0.5371],
        [0.4512, 0.5488],
        [0.4638, 0.5362]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 0.9433 (0.943)  Acc@1: 68.7500 (68.7500)  Acc@5: 100.0000 (100.0000)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 0.4378 (0.615)  Acc@1: 81.2500 (79.1973)  Acc@5: 100.0000 (99.0502)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 0.7298 (0.597)  Acc@1: 65.6250 (79.6720)  Acc@5: 100.0000 (98.8243)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 0.7373 (0.593)  Acc@1: 68.7500 (79.4495)  Acc@5: 100.0000 (98.8411)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 0.6981 (0.592)  Acc@1: 79.6875 (79.3221)  Acc@5: 96.8750 (98.8884)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 0.5457 (0.593)  Acc@1: 82.8125 (79.3638)  Acc@5: 100.0000 (98.8919)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 0.4658 (0.593)  Acc@1: 81.2500 (79.1944)  Acc@5: 98.4375 (98.9047)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 0.4712 (0.595)  Acc@1: 89.0625 (79.1132)  Acc@5: 100.0000 (98.9049)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 0.4920 (0.594)  Acc@1: 85.0000 (79.2200)  Acc@5: 100.0000 (98.9320)LR: 2.386e-02
train_acc 79.220000
Valid: 7 [   0/390]  Loss: 0.9257 (0.926)  Acc@1: 68.7500 (68.7500)  Acc@5: 96.8750 (96.8750)
Valid: 7 [  50/390]  Loss: 0.5838 (0.667)  Acc@1: 79.6875 (77.0833)  Acc@5: 98.4375 (98.2537)
Valid: 7 [ 100/390]  Loss: 0.5698 (0.683)  Acc@1: 75.0000 (76.7481)  Acc@5: 98.4375 (98.1745)
Valid: 7 [ 150/390]  Loss: 0.8171 (0.694)  Acc@1: 73.4375 (76.3969)  Acc@5: 98.4375 (98.0650)
Valid: 7 [ 200/390]  Loss: 0.8315 (0.690)  Acc@1: 71.8750 (76.3759)  Acc@5: 96.8750 (98.1654)
Valid: 7 [ 250/390]  Loss: 0.8597 (0.686)  Acc@1: 76.5625 (76.6248)  Acc@5: 95.3125 (98.1823)
Valid: 7 [ 300/390]  Loss: 0.6581 (0.685)  Acc@1: 78.1250 (76.7546)  Acc@5: 100.0000 (98.1831)
Valid: 7 [ 350/390]  Loss: 0.6575 (0.688)  Acc@1: 84.3750 (76.6560)  Acc@5: 98.4375 (98.1793)
Valid: 7 [ 390/390]  Loss: 0.7591 (0.686)  Acc@1: 77.5000 (76.7480)  Acc@5: 100.0000 (98.2240)
valid_acc 76.748000
epoch = 7   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('max_pool_3x3', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4665, 0.5335],
        [0.4707, 0.5293],
        [0.4488, 0.5512],
        [0.4153, 0.5847],
        [0.4493, 0.5507],
        [0.5221, 0.4779],
        [0.4267, 0.5733],
        [0.4482, 0.5518],
        [0.4119, 0.5881],
        [0.4862, 0.5138],
        [0.5348, 0.4652],
        [0.4779, 0.5221],
        [0.5295, 0.4705],
        [0.4934, 0.5066]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6034, 0.3966],
        [0.5301, 0.4699],
        [0.5935, 0.4065],
        [0.5889, 0.4111],
        [0.4559, 0.5441],
        [0.5895, 0.4105],
        [0.5086, 0.4914],
        [0.4722, 0.5278],
        [0.4736, 0.5264],
        [0.5933, 0.4067],
        [0.5877, 0.4123],
        [0.4571, 0.5429],
        [0.4458, 0.5542],
        [0.4590, 0.5410]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 0.6484 (0.648)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 0.5565 (0.557)  Acc@1: 76.5625 (80.8211)  Acc@5: 100.0000 (98.8358)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 0.5767 (0.564)  Acc@1: 78.1250 (80.6002)  Acc@5: 100.0000 (99.0718)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 0.5706 (0.564)  Acc@1: 79.6875 (80.6291)  Acc@5: 100.0000 (99.0687)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 0.6816 (0.565)  Acc@1: 75.0000 (80.4960)  Acc@5: 96.8750 (99.0438)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 0.5691 (0.567)  Acc@1: 79.6875 (80.3909)  Acc@5: 100.0000 (98.9978)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 0.6380 (0.564)  Acc@1: 79.6875 (80.4869)  Acc@5: 100.0000 (99.0137)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 0.5735 (0.565)  Acc@1: 79.6875 (80.4087)  Acc@5: 100.0000 (99.0830)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 0.7928 (0.564)  Acc@1: 72.5000 (80.3080)  Acc@5: 100.0000 (99.0680)LR: 2.352e-02
train_acc 80.308000
Valid: 8 [   0/390]  Loss: 0.5919 (0.592)  Acc@1: 78.1250 (78.1250)  Acc@5: 100.0000 (100.0000)
Valid: 8 [  50/390]  Loss: 0.5231 (0.605)  Acc@1: 79.6875 (79.0135)  Acc@5: 98.4375 (98.8051)
Valid: 8 [ 100/390]  Loss: 0.7439 (0.617)  Acc@1: 78.1250 (78.4808)  Acc@5: 96.8750 (98.9016)
Valid: 8 [ 150/390]  Loss: 0.4814 (0.623)  Acc@1: 82.8125 (78.3113)  Acc@5: 100.0000 (98.8307)
Valid: 8 [ 200/390]  Loss: 0.3418 (0.620)  Acc@1: 89.0625 (78.5292)  Acc@5: 100.0000 (98.8417)
Valid: 8 [ 250/390]  Loss: 0.9299 (0.619)  Acc@1: 67.1875 (78.4300)  Acc@5: 96.8750 (98.8670)
Valid: 8 [ 300/390]  Loss: 0.4637 (0.615)  Acc@1: 85.9375 (78.6493)  Acc@5: 100.0000 (98.8632)
Valid: 8 [ 350/390]  Loss: 0.6583 (0.621)  Acc@1: 78.1250 (78.4722)  Acc@5: 98.4375 (98.8070)
Valid: 8 [ 390/390]  Loss: 0.7236 (0.620)  Acc@1: 75.0000 (78.5480)  Acc@5: 100.0000 (98.7800)
valid_acc 78.548000
epoch = 8   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('max_pool_3x3', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4643, 0.5357],
        [0.4748, 0.5252],
        [0.4412, 0.5588],
        [0.4064, 0.5936],
        [0.4455, 0.5545],
        [0.5157, 0.4843],
        [0.4222, 0.5778],
        [0.4460, 0.5540],
        [0.4136, 0.5864],
        [0.4799, 0.5201],
        [0.5304, 0.4696],
        [0.4795, 0.5205],
        [0.5314, 0.4686],
        [0.4971, 0.5029]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6127, 0.3873],
        [0.5274, 0.4726],
        [0.6017, 0.3983],
        [0.5956, 0.4044],
        [0.4514, 0.5486],
        [0.5965, 0.4035],
        [0.5049, 0.4951],
        [0.4693, 0.5307],
        [0.4743, 0.5257],
        [0.6007, 0.3993],
        [0.5935, 0.4065],
        [0.4593, 0.5407],
        [0.4466, 0.5534],
        [0.4610, 0.5390]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 0.4877 (0.488)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 0.5104 (0.494)  Acc@1: 76.5625 (82.9657)  Acc@5: 100.0000 (99.3566)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 0.5619 (0.513)  Acc@1: 78.1250 (82.2865)  Acc@5: 100.0000 (99.3193)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 0.5078 (0.522)  Acc@1: 84.3750 (81.9329)  Acc@5: 100.0000 (99.3067)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 0.4837 (0.520)  Acc@1: 82.8125 (82.1129)  Acc@5: 100.0000 (99.2537)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 0.4536 (0.530)  Acc@1: 81.2500 (81.7791)  Acc@5: 98.4375 (99.1409)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 0.4988 (0.531)  Acc@1: 79.6875 (81.7120)  Acc@5: 98.4375 (99.1123)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 0.5034 (0.535)  Acc@1: 81.2500 (81.5304)  Acc@5: 100.0000 (99.0830)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 0.5349 (0.537)  Acc@1: 87.5000 (81.5120)  Acc@5: 95.0000 (99.0720)LR: 2.313e-02
train_acc 81.512000
Valid: 9 [   0/390]  Loss: 0.4371 (0.437)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 9 [  50/390]  Loss: 0.6486 (0.613)  Acc@1: 81.2500 (79.8407)  Acc@5: 100.0000 (98.5294)
Valid: 9 [ 100/390]  Loss: 0.4910 (0.622)  Acc@1: 81.2500 (79.1151)  Acc@5: 100.0000 (98.6231)
Valid: 9 [ 150/390]  Loss: 0.6611 (0.619)  Acc@1: 73.4375 (79.3460)  Acc@5: 100.0000 (98.6858)
Valid: 9 [ 200/390]  Loss: 0.7490 (0.618)  Acc@1: 81.2500 (79.3843)  Acc@5: 96.8750 (98.6863)
Valid: 9 [ 250/390]  Loss: 0.7153 (0.615)  Acc@1: 75.0000 (79.3638)  Acc@5: 98.4375 (98.6803)
Valid: 9 [ 300/390]  Loss: 0.6623 (0.617)  Acc@1: 71.8750 (79.2618)  Acc@5: 98.4375 (98.6971)
Valid: 9 [ 350/390]  Loss: 1.065 (0.617)  Acc@1: 73.4375 (79.2512)  Acc@5: 95.3125 (98.6957)
Valid: 9 [ 390/390]  Loss: 0.5211 (0.616)  Acc@1: 85.0000 (79.2800)  Acc@5: 97.5000 (98.7080)
valid_acc 79.280000
epoch = 9   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('max_pool_3x3', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4638, 0.5362],
        [0.4730, 0.5270],
        [0.4340, 0.5660],
        [0.3987, 0.6013],
        [0.4478, 0.5522],
        [0.5069, 0.4931],
        [0.4110, 0.5890],
        [0.4423, 0.5577],
        [0.4070, 0.5930],
        [0.4797, 0.5203],
        [0.5274, 0.4726],
        [0.4818, 0.5182],
        [0.5347, 0.4653],
        [0.4939, 0.5061]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6146, 0.3854],
        [0.5242, 0.4758],
        [0.6026, 0.3974],
        [0.5997, 0.4003],
        [0.4456, 0.5544],
        [0.5965, 0.4035],
        [0.5060, 0.4940],
        [0.4704, 0.5296],
        [0.4730, 0.5270],
        [0.6008, 0.3992],
        [0.5963, 0.4037],
        [0.4603, 0.5397],
        [0.4444, 0.5556],
        [0.4553, 0.5447]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 0.4072 (0.407)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 0.5422 (0.512)  Acc@1: 79.6875 (82.0159)  Acc@5: 100.0000 (99.2341)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 0.6609 (0.508)  Acc@1: 68.7500 (82.1009)  Acc@5: 100.0000 (99.1491)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 0.4373 (0.505)  Acc@1: 84.3750 (82.2848)  Acc@5: 100.0000 (99.1618)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 0.5816 (0.504)  Acc@1: 78.1250 (82.4238)  Acc@5: 100.0000 (99.2226)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 0.4181 (0.500)  Acc@1: 87.5000 (82.6320)  Acc@5: 100.0000 (99.2219)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 0.5773 (0.505)  Acc@1: 81.2500 (82.5478)  Acc@5: 98.4375 (99.2317)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 0.4606 (0.508)  Acc@1: 84.3750 (82.4964)  Acc@5: 100.0000 (99.1987)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 0.5185 (0.507)  Acc@1: 77.5000 (82.6160)  Acc@5: 100.0000 (99.2320)LR: 2.271e-02
train_acc 82.616000
Valid: 10 [   0/390]  Loss: 0.4258 (0.426)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 10 [  50/390]  Loss: 0.7197 (0.586)  Acc@1: 71.8750 (79.5037)  Acc@5: 100.0000 (98.9890)
Valid: 10 [ 100/390]  Loss: 0.4789 (0.593)  Acc@1: 85.9375 (79.4864)  Acc@5: 98.4375 (98.8243)
Valid: 10 [ 150/390]  Loss: 0.6541 (0.589)  Acc@1: 81.2500 (79.8427)  Acc@5: 100.0000 (98.7893)
Valid: 10 [ 200/390]  Loss: 0.5921 (0.579)  Acc@1: 75.0000 (80.0451)  Acc@5: 98.4375 (98.8573)
Valid: 10 [ 250/390]  Loss: 0.8037 (0.583)  Acc@1: 71.8750 (79.8992)  Acc@5: 95.3125 (98.7861)
Valid: 10 [ 300/390]  Loss: 0.6655 (0.583)  Acc@1: 75.0000 (79.8121)  Acc@5: 100.0000 (98.8684)
Valid: 10 [ 350/390]  Loss: 0.6421 (0.588)  Acc@1: 81.2500 (79.7231)  Acc@5: 96.8750 (98.8827)
Valid: 10 [ 390/390]  Loss: 0.5076 (0.592)  Acc@1: 87.5000 (79.6560)  Acc@5: 97.5000 (98.8560)
valid_acc 79.656000
epoch = 10   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4556, 0.5444],
        [0.4709, 0.5291],
        [0.4256, 0.5744],
        [0.3909, 0.6091],
        [0.4469, 0.5531],
        [0.4999, 0.5001],
        [0.4067, 0.5933],
        [0.4407, 0.5593],
        [0.3960, 0.6040],
        [0.4786, 0.5214],
        [0.5204, 0.4796],
        [0.4801, 0.5199],
        [0.5397, 0.4603],
        [0.4917, 0.5083]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6165, 0.3835],
        [0.5222, 0.4778],
        [0.6041, 0.3959],
        [0.6021, 0.3979],
        [0.4348, 0.5652],
        [0.5989, 0.4011],
        [0.5032, 0.4968],
        [0.4653, 0.5347],
        [0.4719, 0.5281],
        [0.6028, 0.3972],
        [0.6001, 0.3999],
        [0.4606, 0.5394],
        [0.4427, 0.5573],
        [0.4517, 0.5483]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 0.4058 (0.406)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 0.4106 (0.462)  Acc@1: 87.5000 (83.7929)  Acc@5: 100.0000 (99.5711)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 0.4946 (0.477)  Acc@1: 82.8125 (82.9517)  Acc@5: 100.0000 (99.4585)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 0.4723 (0.483)  Acc@1: 81.2500 (82.6780)  Acc@5: 100.0000 (99.4516)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 0.6254 (0.479)  Acc@1: 81.2500 (82.9291)  Acc@5: 98.4375 (99.4403)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 0.4679 (0.485)  Acc@1: 76.5625 (82.7502)  Acc@5: 100.0000 (99.4273)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 0.5048 (0.493)  Acc@1: 78.1250 (82.5789)  Acc@5: 100.0000 (99.3667)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 0.4898 (0.492)  Acc@1: 79.6875 (82.7012)  Acc@5: 100.0000 (99.3456)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 0.5562 (0.495)  Acc@1: 82.5000 (82.6400)  Acc@5: 95.0000 (99.3120)LR: 2.225e-02
train_acc 82.640000
Valid: 11 [   0/390]  Loss: 0.7125 (0.713)  Acc@1: 73.4375 (73.4375)  Acc@5: 98.4375 (98.4375)
Valid: 11 [  50/390]  Loss: 0.6076 (0.612)  Acc@1: 78.1250 (79.0441)  Acc@5: 95.3125 (98.4681)
Valid: 11 [ 100/390]  Loss: 0.8581 (0.614)  Acc@1: 70.3125 (79.0223)  Acc@5: 98.4375 (98.6231)
Valid: 11 [ 150/390]  Loss: 0.8120 (0.613)  Acc@1: 71.8750 (79.0770)  Acc@5: 96.8750 (98.6134)
Valid: 11 [ 200/390]  Loss: 0.7018 (0.605)  Acc@1: 79.6875 (79.4621)  Acc@5: 98.4375 (98.6940)
Valid: 11 [ 250/390]  Loss: 0.4423 (0.609)  Acc@1: 84.3750 (79.3513)  Acc@5: 96.8750 (98.7052)
Valid: 11 [ 300/390]  Loss: 0.5303 (0.609)  Acc@1: 73.4375 (79.1788)  Acc@5: 100.0000 (98.6815)
Valid: 11 [ 350/390]  Loss: 0.4795 (0.609)  Acc@1: 81.2500 (79.1311)  Acc@5: 98.4375 (98.6690)
Valid: 11 [ 390/390]  Loss: 0.5062 (0.608)  Acc@1: 85.0000 (79.2040)  Acc@5: 100.0000 (98.6760)
valid_acc 79.204000
epoch = 11   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4486, 0.5514],
        [0.4643, 0.5357],
        [0.4195, 0.5805],
        [0.3786, 0.6214],
        [0.4426, 0.5574],
        [0.4815, 0.5185],
        [0.4038, 0.5962],
        [0.4356, 0.5644],
        [0.3927, 0.6073],
        [0.4739, 0.5261],
        [0.5180, 0.4820],
        [0.4776, 0.5224],
        [0.5392, 0.4608],
        [0.4907, 0.5093]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6228, 0.3772],
        [0.5215, 0.4785],
        [0.6098, 0.3902],
        [0.6044, 0.3956],
        [0.4280, 0.5720],
        [0.6042, 0.3958],
        [0.5008, 0.4992],
        [0.4593, 0.5407],
        [0.4696, 0.5304],
        [0.6074, 0.3926],
        [0.6020, 0.3980],
        [0.4584, 0.5416],
        [0.4408, 0.5592],
        [0.4487, 0.5513]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 0.4971 (0.497)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 0.3490 (0.445)  Acc@1: 87.5000 (84.5282)  Acc@5: 100.0000 (99.4179)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 0.4386 (0.464)  Acc@1: 81.2500 (83.8026)  Acc@5: 100.0000 (99.4121)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 0.5305 (0.467)  Acc@1: 82.8125 (83.6817)  Acc@5: 100.0000 (99.3067)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 0.4460 (0.462)  Acc@1: 81.2500 (83.9241)  Acc@5: 100.0000 (99.3237)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 0.4300 (0.461)  Acc@1: 82.8125 (83.9766)  Acc@5: 100.0000 (99.3277)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 0.4721 (0.461)  Acc@1: 84.3750 (84.0428)  Acc@5: 100.0000 (99.3459)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 0.2804 (0.465)  Acc@1: 92.1875 (83.9343)  Acc@5: 100.0000 (99.3234)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 0.6396 (0.464)  Acc@1: 75.0000 (83.9720)  Acc@5: 100.0000 (99.3400)LR: 2.175e-02
train_acc 83.972000
Valid: 12 [   0/390]  Loss: 0.5933 (0.593)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 12 [  50/390]  Loss: 0.8772 (0.561)  Acc@1: 75.0000 (81.4032)  Acc@5: 98.4375 (98.7132)
Valid: 12 [ 100/390]  Loss: 0.5764 (0.568)  Acc@1: 76.5625 (81.0953)  Acc@5: 98.4375 (98.9171)
Valid: 12 [ 150/390]  Loss: 0.3433 (0.568)  Acc@1: 89.0625 (80.9292)  Acc@5: 100.0000 (98.9652)
Valid: 12 [ 200/390]  Loss: 0.7606 (0.571)  Acc@1: 75.0000 (80.6903)  Acc@5: 98.4375 (98.9428)
Valid: 12 [ 250/390]  Loss: 0.6786 (0.570)  Acc@1: 81.2500 (80.7395)  Acc@5: 98.4375 (98.9355)
Valid: 12 [ 300/390]  Loss: 0.3828 (0.567)  Acc@1: 81.2500 (80.6530)  Acc@5: 100.0000 (98.9618)
Valid: 12 [ 350/390]  Loss: 0.7731 (0.573)  Acc@1: 70.3125 (80.4621)  Acc@5: 96.8750 (98.9405)
Valid: 12 [ 390/390]  Loss: 0.6568 (0.574)  Acc@1: 80.0000 (80.4640)  Acc@5: 97.5000 (98.9040)
valid_acc 80.464000
epoch = 12   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4482, 0.5518],
        [0.4652, 0.5348],
        [0.4155, 0.5845],
        [0.3738, 0.6262],
        [0.4406, 0.5594],
        [0.4713, 0.5287],
        [0.4019, 0.5981],
        [0.4323, 0.5677],
        [0.3919, 0.6081],
        [0.4741, 0.5259],
        [0.5127, 0.4873],
        [0.4813, 0.5187],
        [0.5409, 0.4591],
        [0.4936, 0.5064]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6264, 0.3736],
        [0.5246, 0.4754],
        [0.6114, 0.3886],
        [0.6061, 0.3939],
        [0.4187, 0.5813],
        [0.6073, 0.3927],
        [0.5006, 0.4994],
        [0.4571, 0.5429],
        [0.4706, 0.5294],
        [0.6093, 0.3907],
        [0.6019, 0.3981],
        [0.4549, 0.5451],
        [0.4369, 0.5631],
        [0.4432, 0.5568]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 0.4060 (0.406)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 0.5451 (0.442)  Acc@1: 76.5625 (84.4056)  Acc@5: 96.8750 (99.3566)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 0.4935 (0.448)  Acc@1: 84.3750 (84.6380)  Acc@5: 100.0000 (99.4276)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 0.4378 (0.446)  Acc@1: 85.9375 (84.7372)  Acc@5: 100.0000 (99.4205)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 0.2695 (0.449)  Acc@1: 92.1875 (84.7326)  Acc@5: 100.0000 (99.4714)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 0.4644 (0.455)  Acc@1: 87.5000 (84.4746)  Acc@5: 100.0000 (99.4709)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 0.5211 (0.454)  Acc@1: 84.3750 (84.4529)  Acc@5: 100.0000 (99.4549)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 0.4496 (0.455)  Acc@1: 79.6875 (84.4195)  Acc@5: 100.0000 (99.4525)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 0.3376 (0.455)  Acc@1: 90.0000 (84.3720)  Acc@5: 100.0000 (99.4400)LR: 2.121e-02
train_acc 84.372000
Valid: 13 [   0/390]  Loss: 0.2957 (0.296)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 13 [  50/390]  Loss: 0.5354 (0.528)  Acc@1: 85.9375 (82.4142)  Acc@5: 98.4375 (99.0196)
Valid: 13 [ 100/390]  Loss: 0.3953 (0.526)  Acc@1: 84.3750 (82.3639)  Acc@5: 100.0000 (99.0254)
Valid: 13 [ 150/390]  Loss: 0.6878 (0.529)  Acc@1: 81.2500 (82.1296)  Acc@5: 98.4375 (98.9963)
Valid: 13 [ 200/390]  Loss: 0.7769 (0.529)  Acc@1: 73.4375 (82.1984)  Acc@5: 98.4375 (98.9894)
Valid: 13 [ 250/390]  Loss: 0.7161 (0.537)  Acc@1: 78.1250 (82.0032)  Acc@5: 100.0000 (98.9666)
Valid: 13 [ 300/390]  Loss: 0.4343 (0.534)  Acc@1: 84.3750 (82.0650)  Acc@5: 98.4375 (98.9514)
Valid: 13 [ 350/390]  Loss: 0.5061 (0.526)  Acc@1: 85.9375 (82.2783)  Acc@5: 100.0000 (98.9806)
Valid: 13 [ 390/390]  Loss: 0.5760 (0.530)  Acc@1: 82.5000 (82.1440)  Acc@5: 100.0000 (98.9400)
valid_acc 82.144000
epoch = 13   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4433, 0.5567],
        [0.4667, 0.5333],
        [0.4088, 0.5912],
        [0.3663, 0.6337],
        [0.4345, 0.5655],
        [0.4637, 0.5363],
        [0.3950, 0.6050],
        [0.4296, 0.5704],
        [0.3865, 0.6135],
        [0.4767, 0.5233],
        [0.5078, 0.4922],
        [0.4833, 0.5167],
        [0.5436, 0.4564],
        [0.4904, 0.5096]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6304, 0.3696],
        [0.5216, 0.4784],
        [0.6131, 0.3869],
        [0.6097, 0.3903],
        [0.4163, 0.5837],
        [0.6094, 0.3906],
        [0.4999, 0.5001],
        [0.4560, 0.5440],
        [0.4687, 0.5313],
        [0.6117, 0.3883],
        [0.6067, 0.3933],
        [0.4515, 0.5485],
        [0.4333, 0.5667],
        [0.4395, 0.5605]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 0.3022 (0.302)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 0.4294 (0.392)  Acc@1: 81.2500 (86.4890)  Acc@5: 98.4375 (99.3873)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 0.3571 (0.411)  Acc@1: 87.5000 (85.8756)  Acc@5: 100.0000 (99.3812)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 0.4242 (0.416)  Acc@1: 82.8125 (85.5029)  Acc@5: 98.4375 (99.3688)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 0.7013 (0.421)  Acc@1: 75.0000 (85.1446)  Acc@5: 95.3125 (99.3859)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 0.3437 (0.425)  Acc@1: 87.5000 (85.0224)  Acc@5: 100.0000 (99.3464)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 0.4053 (0.422)  Acc@1: 84.3750 (85.1225)  Acc@5: 100.0000 (99.3823)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 0.3937 (0.425)  Acc@1: 84.3750 (84.9626)  Acc@5: 100.0000 (99.3901)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 0.4202 (0.428)  Acc@1: 85.0000 (84.9760)  Acc@5: 100.0000 (99.3640)LR: 2.065e-02
train_acc 84.976000
Valid: 14 [   0/390]  Loss: 0.6722 (0.672)  Acc@1: 76.5625 (76.5625)  Acc@5: 100.0000 (100.0000)
Valid: 14 [  50/390]  Loss: 0.5859 (0.592)  Acc@1: 82.8125 (79.8713)  Acc@5: 98.4375 (98.7439)
Valid: 14 [ 100/390]  Loss: 0.4131 (0.582)  Acc@1: 87.5000 (80.7550)  Acc@5: 100.0000 (98.7624)
Valid: 14 [ 150/390]  Loss: 0.6079 (0.577)  Acc@1: 81.2500 (80.5774)  Acc@5: 98.4375 (98.9031)
Valid: 14 [ 200/390]  Loss: 0.8740 (0.582)  Acc@1: 79.6875 (80.5970)  Acc@5: 95.3125 (98.8417)
Valid: 14 [ 250/390]  Loss: 0.5762 (0.581)  Acc@1: 82.8125 (80.7831)  Acc@5: 100.0000 (98.8421)
Valid: 14 [ 300/390]  Loss: 0.5884 (0.583)  Acc@1: 81.2500 (80.7309)  Acc@5: 100.0000 (98.8891)
Valid: 14 [ 350/390]  Loss: 0.4017 (0.586)  Acc@1: 87.5000 (80.7425)  Acc@5: 100.0000 (98.9227)
Valid: 14 [ 390/390]  Loss: 0.2218 (0.589)  Acc@1: 92.5000 (80.6120)  Acc@5: 100.0000 (98.8960)
valid_acc 80.612000
epoch = 14   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4368, 0.5632],
        [0.4576, 0.5424],
        [0.4045, 0.5955],
        [0.3634, 0.6366],
        [0.4304, 0.5696],
        [0.4436, 0.5564],
        [0.3872, 0.6128],
        [0.4249, 0.5751],
        [0.3830, 0.6170],
        [0.4778, 0.5222],
        [0.5029, 0.4971],
        [0.4802, 0.5198],
        [0.5437, 0.4563],
        [0.4927, 0.5073]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6328, 0.3672],
        [0.5201, 0.4799],
        [0.6132, 0.3868],
        [0.6087, 0.3913],
        [0.4098, 0.5902],
        [0.6079, 0.3921],
        [0.4994, 0.5006],
        [0.4537, 0.5463],
        [0.4661, 0.5339],
        [0.6114, 0.3886],
        [0.6057, 0.3943],
        [0.4489, 0.5511],
        [0.4287, 0.5713],
        [0.4339, 0.5661]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 0.4307 (0.431)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 0.3877 (0.380)  Acc@1: 85.9375 (86.8566)  Acc@5: 100.0000 (99.3566)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 0.3134 (0.376)  Acc@1: 90.6250 (86.4790)  Acc@5: 98.4375 (99.4585)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 0.3738 (0.390)  Acc@1: 82.8125 (86.2686)  Acc@5: 100.0000 (99.4412)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 0.3862 (0.398)  Acc@1: 84.3750 (85.9764)  Acc@5: 100.0000 (99.4714)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 0.4853 (0.408)  Acc@1: 81.2500 (85.5515)  Acc@5: 98.4375 (99.4646)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 0.3795 (0.416)  Acc@1: 81.2500 (85.2990)  Acc@5: 100.0000 (99.4549)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 0.3125 (0.418)  Acc@1: 87.5000 (85.3009)  Acc@5: 98.4375 (99.4257)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 0.3088 (0.416)  Acc@1: 90.0000 (85.3800)  Acc@5: 100.0000 (99.4480)LR: 2.005e-02
train_acc 85.380000
Valid: 15 [   0/390]  Loss: 0.4981 (0.498)  Acc@1: 76.5625 (76.5625)  Acc@5: 98.4375 (98.4375)
Valid: 15 [  50/390]  Loss: 0.4496 (0.540)  Acc@1: 82.8125 (81.8321)  Acc@5: 100.0000 (99.1728)
Valid: 15 [ 100/390]  Loss: 0.3442 (0.540)  Acc@1: 87.5000 (82.1627)  Acc@5: 96.8750 (98.9325)
Valid: 15 [ 150/390]  Loss: 0.6779 (0.538)  Acc@1: 76.5625 (82.0778)  Acc@5: 100.0000 (99.0791)
Valid: 15 [ 200/390]  Loss: 0.5428 (0.527)  Acc@1: 82.8125 (82.3305)  Acc@5: 98.4375 (99.1449)
Valid: 15 [ 250/390]  Loss: 0.6278 (0.526)  Acc@1: 81.2500 (82.5261)  Acc@5: 95.3125 (99.1098)
Valid: 15 [ 300/390]  Loss: 0.5709 (0.527)  Acc@1: 84.3750 (82.5789)  Acc@5: 98.4375 (99.1331)
Valid: 15 [ 350/390]  Loss: 0.6224 (0.526)  Acc@1: 82.8125 (82.5677)  Acc@5: 96.8750 (99.1230)
Valid: 15 [ 390/390]  Loss: 0.7198 (0.527)  Acc@1: 70.0000 (82.5560)  Acc@5: 97.5000 (99.0960)
valid_acc 82.556000
epoch = 15   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4360, 0.5640],
        [0.4528, 0.5472],
        [0.3998, 0.6002],
        [0.3564, 0.6436],
        [0.4284, 0.5716],
        [0.4290, 0.5710],
        [0.3793, 0.6207],
        [0.4254, 0.5746],
        [0.3790, 0.6210],
        [0.4846, 0.5154],
        [0.4938, 0.5062],
        [0.4816, 0.5184],
        [0.5505, 0.4495],
        [0.4948, 0.5052]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6319, 0.3681],
        [0.5171, 0.4829],
        [0.6138, 0.3862],
        [0.6091, 0.3909],
        [0.4036, 0.5964],
        [0.6077, 0.3923],
        [0.4938, 0.5062],
        [0.4468, 0.5532],
        [0.4646, 0.5354],
        [0.6128, 0.3872],
        [0.6059, 0.3941],
        [0.4425, 0.5575],
        [0.4234, 0.5766],
        [0.4311, 0.5689]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 0.2120 (0.212)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 0.1994 (0.379)  Acc@1: 93.7500 (86.8566)  Acc@5: 100.0000 (99.5711)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 0.5596 (0.381)  Acc@1: 75.0000 (86.4635)  Acc@5: 96.8750 (99.5823)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 0.3298 (0.385)  Acc@1: 89.0625 (86.4652)  Acc@5: 100.0000 (99.5757)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 0.6190 (0.395)  Acc@1: 79.6875 (86.1474)  Acc@5: 98.4375 (99.5180)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 0.4425 (0.402)  Acc@1: 84.3750 (86.0309)  Acc@5: 100.0000 (99.4460)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 0.5538 (0.404)  Acc@1: 82.8125 (85.8752)  Acc@5: 100.0000 (99.4549)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 0.3191 (0.402)  Acc@1: 92.1875 (85.9598)  Acc@5: 100.0000 (99.4614)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 0.6822 (0.401)  Acc@1: 72.5000 (85.9680)  Acc@5: 100.0000 (99.4800)LR: 1.943e-02
train_acc 85.968000
Valid: 16 [   0/390]  Loss: 0.6934 (0.693)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 16 [  50/390]  Loss: 0.3691 (0.570)  Acc@1: 85.9375 (81.5257)  Acc@5: 98.4375 (98.9583)
Valid: 16 [ 100/390]  Loss: 0.7565 (0.560)  Acc@1: 75.0000 (81.7915)  Acc@5: 96.8750 (98.8707)
Valid: 16 [ 150/390]  Loss: 0.5490 (0.549)  Acc@1: 85.9375 (82.0571)  Acc@5: 100.0000 (98.9445)
Valid: 16 [ 200/390]  Loss: 0.6247 (0.559)  Acc@1: 78.1250 (81.9108)  Acc@5: 100.0000 (98.9195)
Valid: 16 [ 250/390]  Loss: 0.3379 (0.553)  Acc@1: 82.8125 (81.9099)  Acc@5: 100.0000 (98.9791)
Valid: 16 [ 300/390]  Loss: 0.5819 (0.553)  Acc@1: 84.3750 (81.9871)  Acc@5: 100.0000 (98.9514)
Valid: 16 [ 350/390]  Loss: 0.8349 (0.558)  Acc@1: 75.0000 (81.8421)  Acc@5: 96.8750 (98.9583)
Valid: 16 [ 390/390]  Loss: 0.2489 (0.557)  Acc@1: 92.5000 (81.8240)  Acc@5: 100.0000 (98.9920)
valid_acc 81.824000
epoch = 16   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4366, 0.5634],
        [0.4498, 0.5502],
        [0.3960, 0.6040],
        [0.3494, 0.6506],
        [0.4291, 0.5709],
        [0.4110, 0.5890],
        [0.3707, 0.6293],
        [0.4208, 0.5792],
        [0.3795, 0.6205],
        [0.4898, 0.5102],
        [0.4860, 0.5140],
        [0.4834, 0.5166],
        [0.5543, 0.4457],
        [0.4992, 0.5008]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6358, 0.3642],
        [0.5169, 0.4831],
        [0.6188, 0.3812],
        [0.6134, 0.3866],
        [0.4007, 0.5993],
        [0.6115, 0.3885],
        [0.4900, 0.5100],
        [0.4402, 0.5598],
        [0.4593, 0.5407],
        [0.6156, 0.3844],
        [0.6080, 0.3920],
        [0.4379, 0.5621],
        [0.4180, 0.5820],
        [0.4302, 0.5698]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 0.3303 (0.330)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 0.4425 (0.383)  Acc@1: 82.8125 (86.4890)  Acc@5: 98.4375 (99.4792)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 0.3981 (0.388)  Acc@1: 85.9375 (86.5718)  Acc@5: 100.0000 (99.4276)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 0.4968 (0.380)  Acc@1: 84.3750 (87.1482)  Acc@5: 100.0000 (99.4930)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 0.4285 (0.382)  Acc@1: 82.8125 (86.9481)  Acc@5: 100.0000 (99.5180)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 0.3138 (0.386)  Acc@1: 89.0625 (86.8339)  Acc@5: 100.0000 (99.5207)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 0.5698 (0.388)  Acc@1: 81.2500 (86.7681)  Acc@5: 96.8750 (99.5017)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 0.3568 (0.386)  Acc@1: 87.5000 (86.8145)  Acc@5: 100.0000 (99.5192)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 0.4450 (0.388)  Acc@1: 87.5000 (86.7040)  Acc@5: 97.5000 (99.5160)LR: 1.878e-02
train_acc 86.704000
Valid: 17 [   0/390]  Loss: 0.5816 (0.582)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)
Valid: 17 [  50/390]  Loss: 0.6881 (0.590)  Acc@1: 81.2500 (80.3309)  Acc@5: 100.0000 (98.9583)
Valid: 17 [ 100/390]  Loss: 0.4964 (0.576)  Acc@1: 82.8125 (80.8942)  Acc@5: 100.0000 (98.9944)
Valid: 17 [ 150/390]  Loss: 0.5474 (0.577)  Acc@1: 85.9375 (80.8878)  Acc@5: 100.0000 (99.0687)
Valid: 17 [ 200/390]  Loss: 0.5162 (0.585)  Acc@1: 87.5000 (80.7214)  Acc@5: 98.4375 (98.9506)
Valid: 17 [ 250/390]  Loss: 0.4021 (0.575)  Acc@1: 82.8125 (81.0632)  Acc@5: 100.0000 (99.0102)
Valid: 17 [ 300/390]  Loss: 0.7759 (0.571)  Acc@1: 79.6875 (81.2863)  Acc@5: 96.8750 (99.0189)
Valid: 17 [ 350/390]  Loss: 0.6469 (0.574)  Acc@1: 75.0000 (81.2455)  Acc@5: 100.0000 (99.0207)
Valid: 17 [ 390/390]  Loss: 0.8781 (0.572)  Acc@1: 75.0000 (81.2320)  Acc@5: 95.0000 (99.0520)
valid_acc 81.232000
epoch = 17   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 3), ('skip_connect', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4340, 0.5660],
        [0.4437, 0.5563],
        [0.3898, 0.6102],
        [0.3451, 0.6549],
        [0.4268, 0.5732],
        [0.3952, 0.6048],
        [0.3687, 0.6313],
        [0.4147, 0.5853],
        [0.3773, 0.6227],
        [0.4888, 0.5112],
        [0.4782, 0.5218],
        [0.4847, 0.5153],
        [0.5597, 0.4403],
        [0.5000, 0.5000]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6380, 0.3620],
        [0.5124, 0.4876],
        [0.6217, 0.3783],
        [0.6129, 0.3871],
        [0.3938, 0.6062],
        [0.6141, 0.3859],
        [0.4802, 0.5198],
        [0.4382, 0.5618],
        [0.4585, 0.5415],
        [0.6203, 0.3797],
        [0.6070, 0.3930],
        [0.4359, 0.5641],
        [0.4175, 0.5825],
        [0.4277, 0.5723]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 0.2613 (0.261)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 0.5063 (0.348)  Acc@1: 81.2500 (88.1740)  Acc@5: 98.4375 (99.5711)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 0.4702 (0.358)  Acc@1: 84.3750 (87.2989)  Acc@5: 98.4375 (99.5668)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 0.5036 (0.361)  Acc@1: 82.8125 (87.3655)  Acc@5: 100.0000 (99.4930)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 0.5311 (0.367)  Acc@1: 81.2500 (87.0336)  Acc@5: 100.0000 (99.5336)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 0.5192 (0.367)  Acc@1: 79.6875 (87.0144)  Acc@5: 98.4375 (99.4958)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 0.4441 (0.373)  Acc@1: 87.5000 (86.8563)  Acc@5: 100.0000 (99.5380)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 0.4433 (0.375)  Acc@1: 85.9375 (86.8011)  Acc@5: 98.4375 (99.5192)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 0.4086 (0.376)  Acc@1: 82.5000 (86.7800)  Acc@5: 100.0000 (99.5320)LR: 1.811e-02
train_acc 86.780000
Valid: 18 [   0/390]  Loss: 0.5068 (0.507)  Acc@1: 81.2500 (81.2500)  Acc@5: 96.8750 (96.8750)
Valid: 18 [  50/390]  Loss: 0.4260 (0.499)  Acc@1: 87.5000 (83.8542)  Acc@5: 100.0000 (99.1422)
Valid: 18 [ 100/390]  Loss: 0.6675 (0.523)  Acc@1: 79.6875 (82.9053)  Acc@5: 98.4375 (99.1027)
Valid: 18 [ 150/390]  Loss: 0.4825 (0.529)  Acc@1: 81.2500 (82.7090)  Acc@5: 96.8750 (99.0791)
Valid: 18 [ 200/390]  Loss: 0.2728 (0.527)  Acc@1: 93.7500 (82.7503)  Acc@5: 100.0000 (99.0361)
Valid: 18 [ 250/390]  Loss: 0.4732 (0.534)  Acc@1: 84.3750 (82.4452)  Acc@5: 100.0000 (99.0289)
Valid: 18 [ 300/390]  Loss: 0.5024 (0.529)  Acc@1: 84.3750 (82.6879)  Acc@5: 98.4375 (99.0397)
Valid: 18 [ 350/390]  Loss: 0.7003 (0.530)  Acc@1: 75.0000 (82.5899)  Acc@5: 98.4375 (99.0296)
Valid: 18 [ 390/390]  Loss: 0.5836 (0.530)  Acc@1: 85.0000 (82.6520)  Acc@5: 100.0000 (99.0560)
valid_acc 82.652000
epoch = 18   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 3), ('skip_connect', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4261, 0.5739],
        [0.4392, 0.5608],
        [0.3825, 0.6175],
        [0.3391, 0.6609],
        [0.4231, 0.5769],
        [0.3812, 0.6188],
        [0.3668, 0.6332],
        [0.4121, 0.5879],
        [0.3771, 0.6229],
        [0.4858, 0.5142],
        [0.4669, 0.5331],
        [0.4841, 0.5159],
        [0.5592, 0.4408],
        [0.4968, 0.5032]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6410, 0.3590],
        [0.5115, 0.4885],
        [0.6223, 0.3777],
        [0.6110, 0.3890],
        [0.3874, 0.6126],
        [0.6152, 0.3848],
        [0.4753, 0.5247],
        [0.4340, 0.5660],
        [0.4582, 0.5418],
        [0.6218, 0.3782],
        [0.6044, 0.3956],
        [0.4303, 0.5697],
        [0.4169, 0.5831],
        [0.4228, 0.5772]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 0.3597 (0.360)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 0.3244 (0.338)  Acc@1: 92.1875 (87.9902)  Acc@5: 100.0000 (99.7243)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 0.5096 (0.340)  Acc@1: 84.3750 (87.8094)  Acc@5: 98.4375 (99.7061)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 0.5284 (0.349)  Acc@1: 82.8125 (87.7173)  Acc@5: 98.4375 (99.6482)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 0.4200 (0.359)  Acc@1: 78.1250 (87.5233)  Acc@5: 100.0000 (99.6424)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 0.4416 (0.358)  Acc@1: 89.0625 (87.6494)  Acc@5: 96.8750 (99.6389)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 0.2543 (0.360)  Acc@1: 89.0625 (87.5779)  Acc@5: 100.0000 (99.6366)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 0.4929 (0.362)  Acc@1: 82.8125 (87.5223)  Acc@5: 100.0000 (99.6172)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 0.4491 (0.364)  Acc@1: 85.0000 (87.4680)  Acc@5: 100.0000 (99.6040)LR: 1.742e-02
train_acc 87.468000
Valid: 19 [   0/390]  Loss: 0.3129 (0.313)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)
Valid: 19 [  50/390]  Loss: 0.6263 (0.562)  Acc@1: 81.2500 (82.0466)  Acc@5: 98.4375 (98.9583)
Valid: 19 [ 100/390]  Loss: 0.4813 (0.559)  Acc@1: 84.3750 (81.6986)  Acc@5: 98.4375 (99.0408)
Valid: 19 [ 150/390]  Loss: 0.4116 (0.552)  Acc@1: 87.5000 (82.1192)  Acc@5: 100.0000 (99.0480)
Valid: 19 [ 200/390]  Loss: 0.3208 (0.549)  Acc@1: 89.0625 (82.1751)  Acc@5: 100.0000 (99.0594)
Valid: 19 [ 250/390]  Loss: 0.5729 (0.548)  Acc@1: 79.6875 (82.2709)  Acc@5: 98.4375 (99.0289)
Valid: 19 [ 300/390]  Loss: 0.5842 (0.546)  Acc@1: 82.8125 (82.3453)  Acc@5: 100.0000 (99.0708)
Valid: 19 [ 350/390]  Loss: 0.4120 (0.545)  Acc@1: 79.6875 (82.3629)  Acc@5: 100.0000 (99.0563)
Valid: 19 [ 390/390]  Loss: 0.6645 (0.543)  Acc@1: 85.0000 (82.3440)  Acc@5: 100.0000 (99.0560)
valid_acc 82.344000
epoch = 19   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4224, 0.5776],
        [0.4405, 0.5595],
        [0.3760, 0.6240],
        [0.3340, 0.6660],
        [0.4232, 0.5768],
        [0.3668, 0.6332],
        [0.3690, 0.6310],
        [0.4126, 0.5874],
        [0.3776, 0.6224],
        [0.4860, 0.5140],
        [0.4503, 0.5497],
        [0.4906, 0.5094],
        [0.5678, 0.4322],
        [0.4945, 0.5055]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6421, 0.3579],
        [0.5070, 0.4930],
        [0.6235, 0.3765],
        [0.6118, 0.3882],
        [0.3826, 0.6174],
        [0.6159, 0.3841],
        [0.4711, 0.5289],
        [0.4340, 0.5660],
        [0.4523, 0.5477],
        [0.6241, 0.3759],
        [0.6054, 0.3946],
        [0.4247, 0.5753],
        [0.4170, 0.5830],
        [0.4202, 0.5798]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 0.3569 (0.357)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 0.4060 (0.344)  Acc@1: 84.3750 (88.3272)  Acc@5: 100.0000 (99.4179)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 0.3980 (0.338)  Acc@1: 87.5000 (88.3973)  Acc@5: 98.4375 (99.5668)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 0.5508 (0.343)  Acc@1: 85.9375 (88.2554)  Acc@5: 98.4375 (99.5861)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 0.3024 (0.341)  Acc@1: 89.0625 (88.2696)  Acc@5: 100.0000 (99.5880)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 0.2191 (0.344)  Acc@1: 90.6250 (88.0416)  Acc@5: 100.0000 (99.5891)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 0.3243 (0.346)  Acc@1: 89.0625 (87.8841)  Acc@5: 100.0000 (99.6107)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 0.4318 (0.349)  Acc@1: 84.3750 (87.8428)  Acc@5: 100.0000 (99.6127)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 0.4885 (0.348)  Acc@1: 80.0000 (87.8000)  Acc@5: 100.0000 (99.6120)LR: 1.671e-02
train_acc 87.800000
Valid: 20 [   0/390]  Loss: 0.4520 (0.452)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 20 [  50/390]  Loss: 0.7851 (0.563)  Acc@1: 76.5625 (81.8015)  Acc@5: 96.8750 (99.0196)
Valid: 20 [ 100/390]  Loss: 0.4890 (0.545)  Acc@1: 85.9375 (82.1937)  Acc@5: 98.4375 (99.1027)
Valid: 20 [ 150/390]  Loss: 0.8277 (0.545)  Acc@1: 75.0000 (82.2020)  Acc@5: 100.0000 (99.1722)
Valid: 20 [ 200/390]  Loss: 0.4524 (0.545)  Acc@1: 85.9375 (82.2683)  Acc@5: 96.8750 (99.1449)
Valid: 20 [ 250/390]  Loss: 0.5704 (0.546)  Acc@1: 81.2500 (82.0655)  Acc@5: 100.0000 (99.2032)
Valid: 20 [ 300/390]  Loss: 0.6720 (0.543)  Acc@1: 79.6875 (82.2259)  Acc@5: 98.4375 (99.2317)
Valid: 20 [ 350/390]  Loss: 0.6934 (0.549)  Acc@1: 79.6875 (82.0157)  Acc@5: 100.0000 (99.2076)
Valid: 20 [ 390/390]  Loss: 0.4368 (0.546)  Acc@1: 82.5000 (82.0920)  Acc@5: 100.0000 (99.2360)
valid_acc 82.092000
epoch = 20   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4174, 0.5826],
        [0.4381, 0.5619],
        [0.3706, 0.6294],
        [0.3282, 0.6718],
        [0.4182, 0.5818],
        [0.3533, 0.6467],
        [0.3680, 0.6320],
        [0.4138, 0.5862],
        [0.3795, 0.6205],
        [0.4851, 0.5149],
        [0.4351, 0.5649],
        [0.4915, 0.5085],
        [0.5696, 0.4304],
        [0.4921, 0.5079]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6473, 0.3527],
        [0.5029, 0.4971],
        [0.6282, 0.3718],
        [0.6132, 0.3868],
        [0.3788, 0.6212],
        [0.6205, 0.3795],
        [0.4654, 0.5346],
        [0.4296, 0.5704],
        [0.4493, 0.5507],
        [0.6289, 0.3711],
        [0.6058, 0.3942],
        [0.4205, 0.5795],
        [0.4213, 0.5787],
        [0.4202, 0.5798]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 0.1489 (0.149)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 0.7908 (0.336)  Acc@1: 73.4375 (88.8480)  Acc@5: 98.4375 (99.6630)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 0.3191 (0.333)  Acc@1: 93.7500 (88.6448)  Acc@5: 100.0000 (99.7061)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 0.3922 (0.329)  Acc@1: 87.5000 (88.6072)  Acc@5: 100.0000 (99.7103)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 0.4724 (0.335)  Acc@1: 82.8125 (88.3551)  Acc@5: 100.0000 (99.7201)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 0.3835 (0.335)  Acc@1: 89.0625 (88.2906)  Acc@5: 100.0000 (99.7261)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 0.5316 (0.335)  Acc@1: 84.3750 (88.3825)  Acc@5: 100.0000 (99.7197)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 0.4091 (0.335)  Acc@1: 82.8125 (88.3948)  Acc@5: 100.0000 (99.7196)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 0.3821 (0.336)  Acc@1: 82.5000 (88.3320)  Acc@5: 97.5000 (99.7120)LR: 1.598e-02
train_acc 88.332000
Valid: 21 [   0/390]  Loss: 0.5194 (0.519)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 21 [  50/390]  Loss: 0.6461 (0.498)  Acc@1: 78.1250 (83.8848)  Acc@5: 98.4375 (99.1422)
Valid: 21 [ 100/390]  Loss: 0.3869 (0.513)  Acc@1: 89.0625 (83.4158)  Acc@5: 100.0000 (99.1182)
Valid: 21 [ 150/390]  Loss: 0.3781 (0.511)  Acc@1: 87.5000 (83.4644)  Acc@5: 98.4375 (99.1618)
Valid: 21 [ 200/390]  Loss: 0.3491 (0.503)  Acc@1: 92.1875 (83.5976)  Acc@5: 98.4375 (99.1604)
Valid: 21 [ 250/390]  Loss: 0.4015 (0.505)  Acc@1: 81.2500 (83.5346)  Acc@5: 100.0000 (99.1596)
Valid: 21 [ 300/390]  Loss: 0.4650 (0.505)  Acc@1: 84.3750 (83.5756)  Acc@5: 100.0000 (99.1954)
Valid: 21 [ 350/390]  Loss: 0.6400 (0.508)  Acc@1: 82.8125 (83.5693)  Acc@5: 100.0000 (99.1898)
Valid: 21 [ 390/390]  Loss: 0.4835 (0.509)  Acc@1: 82.5000 (83.5240)  Acc@5: 100.0000 (99.1760)
valid_acc 83.524000
epoch = 21   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4157, 0.5843],
        [0.4331, 0.5669],
        [0.3647, 0.6353],
        [0.3233, 0.6767],
        [0.4142, 0.5858],
        [0.3391, 0.6609],
        [0.3660, 0.6340],
        [0.4073, 0.5927],
        [0.3825, 0.6175],
        [0.4848, 0.5152],
        [0.4270, 0.5730],
        [0.4912, 0.5088],
        [0.5699, 0.4301],
        [0.4897, 0.5103]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6496, 0.3504],
        [0.4995, 0.5005],
        [0.6292, 0.3708],
        [0.6138, 0.3862],
        [0.3781, 0.6219],
        [0.6216, 0.3784],
        [0.4662, 0.5338],
        [0.4298, 0.5702],
        [0.4512, 0.5488],
        [0.6296, 0.3704],
        [0.6069, 0.3931],
        [0.4170, 0.5830],
        [0.4222, 0.5778],
        [0.4140, 0.5860]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.3094 (0.309)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 0.2670 (0.310)  Acc@1: 90.6250 (89.4301)  Acc@5: 100.0000 (99.8162)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 0.3028 (0.319)  Acc@1: 89.0625 (88.7376)  Acc@5: 100.0000 (99.7834)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 0.3586 (0.320)  Acc@1: 87.5000 (88.6693)  Acc@5: 100.0000 (99.8137)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.4683 (0.326)  Acc@1: 82.8125 (88.5261)  Acc@5: 100.0000 (99.7823)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 0.1920 (0.328)  Acc@1: 95.3125 (88.5334)  Acc@5: 100.0000 (99.7634)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 0.2138 (0.325)  Acc@1: 92.1875 (88.6628)  Acc@5: 100.0000 (99.7508)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 0.2507 (0.328)  Acc@1: 92.1875 (88.6173)  Acc@5: 100.0000 (99.7374)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 0.3392 (0.329)  Acc@1: 82.5000 (88.5520)  Acc@5: 100.0000 (99.7280)LR: 1.525e-02
train_acc 88.552000
Valid: 22 [   0/390]  Loss: 0.6423 (0.642)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 22 [  50/390]  Loss: 0.4388 (0.527)  Acc@1: 79.6875 (82.6900)  Acc@5: 100.0000 (99.0809)
Valid: 22 [ 100/390]  Loss: 0.7831 (0.501)  Acc@1: 78.1250 (83.6015)  Acc@5: 96.8750 (99.0718)
Valid: 22 [ 150/390]  Loss: 0.3487 (0.512)  Acc@1: 84.3750 (83.3609)  Acc@5: 100.0000 (99.0066)
Valid: 22 [ 200/390]  Loss: 0.5908 (0.527)  Acc@1: 82.8125 (83.0146)  Acc@5: 98.4375 (98.9972)
Valid: 22 [ 250/390]  Loss: 0.1902 (0.526)  Acc@1: 89.0625 (83.0117)  Acc@5: 100.0000 (99.0600)
Valid: 22 [ 300/390]  Loss: 0.4945 (0.523)  Acc@1: 85.9375 (83.0357)  Acc@5: 98.4375 (99.0656)
Valid: 22 [ 350/390]  Loss: 0.6435 (0.520)  Acc@1: 78.1250 (83.2042)  Acc@5: 96.8750 (99.0652)
Valid: 22 [ 390/390]  Loss: 0.8501 (0.521)  Acc@1: 72.5000 (83.1200)  Acc@5: 100.0000 (99.0720)
valid_acc 83.120000
epoch = 22   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4124, 0.5876],
        [0.4332, 0.5668],
        [0.3556, 0.6444],
        [0.3195, 0.6805],
        [0.4112, 0.5888],
        [0.3274, 0.6726],
        [0.3648, 0.6352],
        [0.4068, 0.5932],
        [0.3892, 0.6108],
        [0.4915, 0.5085],
        [0.4170, 0.5830],
        [0.4927, 0.5073],
        [0.5726, 0.4274],
        [0.4892, 0.5108]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6485, 0.3515],
        [0.4950, 0.5050],
        [0.6291, 0.3709],
        [0.6143, 0.3857],
        [0.3732, 0.6268],
        [0.6228, 0.3772],
        [0.4625, 0.5375],
        [0.4337, 0.5663],
        [0.4506, 0.5494],
        [0.6298, 0.3702],
        [0.6068, 0.3932],
        [0.4163, 0.5837],
        [0.4234, 0.5766],
        [0.4159, 0.5841]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 0.3420 (0.342)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 0.2906 (0.332)  Acc@1: 89.0625 (88.3885)  Acc@5: 100.0000 (99.7855)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 0.2258 (0.327)  Acc@1: 90.6250 (88.5520)  Acc@5: 100.0000 (99.8144)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 0.1456 (0.314)  Acc@1: 98.4375 (89.1556)  Acc@5: 100.0000 (99.7827)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 0.3073 (0.310)  Acc@1: 87.5000 (89.2102)  Acc@5: 100.0000 (99.7823)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 0.4756 (0.314)  Acc@1: 81.2500 (89.1310)  Acc@5: 96.8750 (99.7448)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.3702 (0.313)  Acc@1: 87.5000 (89.2598)  Acc@5: 100.0000 (99.7353)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 0.3120 (0.312)  Acc@1: 90.6250 (89.1293)  Acc@5: 98.4375 (99.7507)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 0.4316 (0.315)  Acc@1: 80.0000 (89.0160)  Acc@5: 100.0000 (99.7520)LR: 1.450e-02
train_acc 89.016000
Valid: 23 [   0/390]  Loss: 0.5203 (0.520)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 23 [  50/390]  Loss: 0.3217 (0.503)  Acc@1: 87.5000 (84.0074)  Acc@5: 100.0000 (99.3873)
Valid: 23 [ 100/390]  Loss: 0.5237 (0.519)  Acc@1: 82.8125 (83.6170)  Acc@5: 100.0000 (99.1955)
Valid: 23 [ 150/390]  Loss: 0.4967 (0.513)  Acc@1: 81.2500 (83.7852)  Acc@5: 98.4375 (99.1722)
Valid: 23 [ 200/390]  Loss: 0.6185 (0.516)  Acc@1: 81.2500 (83.6210)  Acc@5: 100.0000 (99.1838)
Valid: 23 [ 250/390]  Loss: 0.4847 (0.523)  Acc@1: 82.8125 (83.5035)  Acc@5: 98.4375 (99.1721)
Valid: 23 [ 300/390]  Loss: 0.3968 (0.522)  Acc@1: 79.6875 (83.5652)  Acc@5: 100.0000 (99.1798)
Valid: 23 [ 350/390]  Loss: 0.7293 (0.523)  Acc@1: 81.2500 (83.6093)  Acc@5: 100.0000 (99.1542)
Valid: 23 [ 390/390]  Loss: 0.2461 (0.518)  Acc@1: 90.0000 (83.7520)  Acc@5: 100.0000 (99.1840)
valid_acc 83.752000
epoch = 23   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4112, 0.5888],
        [0.4274, 0.5726],
        [0.3504, 0.6496],
        [0.3156, 0.6844],
        [0.4108, 0.5892],
        [0.3130, 0.6870],
        [0.3679, 0.6321],
        [0.4125, 0.5875],
        [0.3950, 0.6050],
        [0.4899, 0.5101],
        [0.4038, 0.5962],
        [0.4998, 0.5002],
        [0.5749, 0.4251],
        [0.4909, 0.5091]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6462, 0.3538],
        [0.4907, 0.5093],
        [0.6266, 0.3734],
        [0.6119, 0.3881],
        [0.3685, 0.6315],
        [0.6203, 0.3797],
        [0.4598, 0.5402],
        [0.4349, 0.5651],
        [0.4467, 0.5533],
        [0.6284, 0.3716],
        [0.6046, 0.3954],
        [0.4158, 0.5842],
        [0.4209, 0.5791],
        [0.4161, 0.5839]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.4507 (0.451)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 0.2951 (0.304)  Acc@1: 90.6250 (89.5221)  Acc@5: 100.0000 (99.8468)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.2863 (0.298)  Acc@1: 90.6250 (89.8515)  Acc@5: 98.4375 (99.7679)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 0.2523 (0.292)  Acc@1: 93.7500 (89.9834)  Acc@5: 100.0000 (99.7517)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 0.4118 (0.294)  Acc@1: 87.5000 (89.9331)  Acc@5: 100.0000 (99.7590)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 0.2880 (0.301)  Acc@1: 89.0625 (89.7099)  Acc@5: 100.0000 (99.7634)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 0.2813 (0.301)  Acc@1: 89.0625 (89.5816)  Acc@5: 100.0000 (99.7404)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 0.3527 (0.303)  Acc@1: 89.0625 (89.5344)  Acc@5: 100.0000 (99.7151)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 0.5885 (0.306)  Acc@1: 80.0000 (89.4440)  Acc@5: 97.5000 (99.7000)LR: 1.375e-02
train_acc 89.444000
Valid: 24 [   0/390]  Loss: 0.4843 (0.484)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 24 [  50/390]  Loss: 0.3141 (0.530)  Acc@1: 89.0625 (83.0576)  Acc@5: 100.0000 (98.9583)
Valid: 24 [ 100/390]  Loss: 0.5308 (0.550)  Acc@1: 85.9375 (82.6733)  Acc@5: 98.4375 (99.0873)
Valid: 24 [ 150/390]  Loss: 0.3674 (0.534)  Acc@1: 92.1875 (83.1022)  Acc@5: 100.0000 (99.1308)
Valid: 24 [ 200/390]  Loss: 0.5863 (0.531)  Acc@1: 85.9375 (83.1079)  Acc@5: 100.0000 (99.1371)
Valid: 24 [ 250/390]  Loss: 0.7553 (0.531)  Acc@1: 81.2500 (83.2918)  Acc@5: 96.8750 (99.0974)
Valid: 24 [ 300/390]  Loss: 0.6422 (0.537)  Acc@1: 82.8125 (83.2122)  Acc@5: 100.0000 (99.1020)
Valid: 24 [ 350/390]  Loss: 0.7977 (0.538)  Acc@1: 75.0000 (83.2532)  Acc@5: 98.4375 (99.0830)
Valid: 24 [ 390/390]  Loss: 0.4837 (0.538)  Acc@1: 82.5000 (83.2200)  Acc@5: 100.0000 (99.1160)
valid_acc 83.220000
epoch = 24   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4093, 0.5907],
        [0.4231, 0.5769],
        [0.3402, 0.6598],
        [0.3151, 0.6849],
        [0.4012, 0.5988],
        [0.2996, 0.7004],
        [0.3639, 0.6361],
        [0.4090, 0.5910],
        [0.3951, 0.6049],
        [0.4933, 0.5067],
        [0.3917, 0.6083],
        [0.4951, 0.5049],
        [0.5717, 0.4283],
        [0.4926, 0.5074]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6471, 0.3529],
        [0.4872, 0.5128],
        [0.6272, 0.3728],
        [0.6097, 0.3903],
        [0.3633, 0.6367],
        [0.6221, 0.3779],
        [0.4580, 0.5420],
        [0.4308, 0.5692],
        [0.4488, 0.5512],
        [0.6311, 0.3689],
        [0.6028, 0.3972],
        [0.4171, 0.5829],
        [0.4205, 0.5795],
        [0.4146, 0.5854]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 0.3671 (0.367)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.2795 (0.304)  Acc@1: 89.0625 (89.4608)  Acc@5: 100.0000 (99.7855)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.2747 (0.299)  Acc@1: 89.0625 (89.2946)  Acc@5: 100.0000 (99.7679)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.3517 (0.296)  Acc@1: 89.0625 (89.5385)  Acc@5: 100.0000 (99.7930)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 0.2012 (0.295)  Acc@1: 93.7500 (89.5056)  Acc@5: 100.0000 (99.7901)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.1631 (0.292)  Acc@1: 93.7500 (89.7037)  Acc@5: 100.0000 (99.7883)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 0.3868 (0.286)  Acc@1: 84.3750 (89.8827)  Acc@5: 100.0000 (99.7872)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 0.5209 (0.287)  Acc@1: 85.9375 (89.8816)  Acc@5: 100.0000 (99.7908)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 0.5409 (0.291)  Acc@1: 87.5000 (89.8080)  Acc@5: 97.5000 (99.7960)LR: 1.300e-02
train_acc 89.808000
Valid: 25 [   0/390]  Loss: 0.3409 (0.341)  Acc@1: 93.7500 (93.7500)  Acc@5: 98.4375 (98.4375)
Valid: 25 [  50/390]  Loss: 0.5658 (0.546)  Acc@1: 85.9375 (82.1691)  Acc@5: 98.4375 (99.1728)
Valid: 25 [ 100/390]  Loss: 0.5132 (0.543)  Acc@1: 82.8125 (82.8589)  Acc@5: 100.0000 (99.1801)
Valid: 25 [ 150/390]  Loss: 0.3475 (0.542)  Acc@1: 85.9375 (82.9781)  Acc@5: 100.0000 (99.1929)
Valid: 25 [ 200/390]  Loss: 0.5310 (0.536)  Acc@1: 78.1250 (83.0379)  Acc@5: 98.4375 (99.1993)
Valid: 25 [ 250/390]  Loss: 0.4890 (0.535)  Acc@1: 85.9375 (83.1985)  Acc@5: 100.0000 (99.1845)
Valid: 25 [ 300/390]  Loss: 0.7448 (0.528)  Acc@1: 78.1250 (83.3731)  Acc@5: 100.0000 (99.1902)
Valid: 25 [ 350/390]  Loss: 0.4749 (0.523)  Acc@1: 84.3750 (83.4268)  Acc@5: 98.4375 (99.2076)
Valid: 25 [ 390/390]  Loss: 0.4573 (0.521)  Acc@1: 85.0000 (83.4400)  Acc@5: 100.0000 (99.2120)
valid_acc 83.440000
epoch = 25   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4108, 0.5892],
        [0.4224, 0.5776],
        [0.3356, 0.6644],
        [0.3161, 0.6839],
        [0.4004, 0.5996],
        [0.2883, 0.7117],
        [0.3655, 0.6345],
        [0.4045, 0.5955],
        [0.3994, 0.6006],
        [0.4981, 0.5019],
        [0.3785, 0.6215],
        [0.4941, 0.5059],
        [0.5760, 0.4240],
        [0.4895, 0.5105]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6484, 0.3516],
        [0.4804, 0.5196],
        [0.6294, 0.3706],
        [0.6107, 0.3893],
        [0.3620, 0.6380],
        [0.6250, 0.3750],
        [0.4501, 0.5499],
        [0.4274, 0.5726],
        [0.4450, 0.5550],
        [0.6335, 0.3665],
        [0.6038, 0.3962],
        [0.4142, 0.5858],
        [0.4203, 0.5797],
        [0.4106, 0.5894]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 0.4300 (0.430)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.2710 (0.284)  Acc@1: 90.6250 (89.9816)  Acc@5: 98.4375 (99.7855)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.1445 (0.267)  Acc@1: 95.3125 (90.4548)  Acc@5: 100.0000 (99.8144)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 0.3674 (0.264)  Acc@1: 92.1875 (90.6871)  Acc@5: 98.4375 (99.8137)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 0.2671 (0.265)  Acc@1: 87.5000 (90.7572)  Acc@5: 100.0000 (99.8134)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 0.3332 (0.277)  Acc@1: 92.1875 (90.3200)  Acc@5: 100.0000 (99.8070)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.3491 (0.279)  Acc@1: 85.9375 (90.2512)  Acc@5: 100.0000 (99.7872)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 0.1809 (0.280)  Acc@1: 92.1875 (90.2377)  Acc@5: 100.0000 (99.7908)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 0.3342 (0.281)  Acc@1: 90.0000 (90.2320)  Acc@5: 100.0000 (99.7800)LR: 1.225e-02
train_acc 90.232000
Valid: 26 [   0/390]  Loss: 0.3964 (0.396)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 26 [  50/390]  Loss: 0.6389 (0.533)  Acc@1: 85.9375 (82.8125)  Acc@5: 96.8750 (99.0196)
Valid: 26 [ 100/390]  Loss: 0.6631 (0.546)  Acc@1: 78.1250 (82.8280)  Acc@5: 100.0000 (99.0563)
Valid: 26 [ 150/390]  Loss: 0.4308 (0.532)  Acc@1: 82.8125 (83.2368)  Acc@5: 100.0000 (99.1618)
Valid: 26 [ 200/390]  Loss: 0.5664 (0.536)  Acc@1: 79.6875 (83.1468)  Acc@5: 100.0000 (99.0983)
Valid: 26 [ 250/390]  Loss: 0.5089 (0.538)  Acc@1: 81.2500 (83.0989)  Acc@5: 98.4375 (99.0787)
Valid: 26 [ 300/390]  Loss: 0.9360 (0.541)  Acc@1: 79.6875 (82.9942)  Acc@5: 100.0000 (99.1071)
Valid: 26 [ 350/390]  Loss: 0.6911 (0.543)  Acc@1: 82.8125 (82.9906)  Acc@5: 100.0000 (99.1453)
Valid: 26 [ 390/390]  Loss: 0.5591 (0.545)  Acc@1: 85.0000 (83.0560)  Acc@5: 100.0000 (99.1240)
valid_acc 83.056000
epoch = 26   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 1)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4018, 0.5982],
        [0.4191, 0.5809],
        [0.3262, 0.6738],
        [0.3131, 0.6869],
        [0.3961, 0.6039],
        [0.2756, 0.7244],
        [0.3628, 0.6372],
        [0.4001, 0.5999],
        [0.3962, 0.6038],
        [0.4954, 0.5046],
        [0.3670, 0.6330],
        [0.4939, 0.5061],
        [0.5784, 0.4216],
        [0.4854, 0.5146]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6480, 0.3520],
        [0.4774, 0.5226],
        [0.6318, 0.3682],
        [0.6087, 0.3913],
        [0.3573, 0.6427],
        [0.6272, 0.3728],
        [0.4515, 0.5485],
        [0.4231, 0.5769],
        [0.4382, 0.5618],
        [0.6358, 0.3642],
        [0.6031, 0.3969],
        [0.4042, 0.5958],
        [0.4111, 0.5889],
        [0.3978, 0.6022]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.3776 (0.378)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.2342 (0.245)  Acc@1: 92.1875 (91.8811)  Acc@5: 100.0000 (99.8162)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 0.1877 (0.256)  Acc@1: 93.7500 (90.9499)  Acc@5: 100.0000 (99.7989)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.2120 (0.259)  Acc@1: 92.1875 (90.7595)  Acc@5: 98.4375 (99.7930)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.2259 (0.273)  Acc@1: 90.6250 (90.3063)  Acc@5: 100.0000 (99.7823)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 0.09760 (0.274)  Acc@1: 98.4375 (90.3511)  Acc@5: 100.0000 (99.7697)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.3494 (0.275)  Acc@1: 90.6250 (90.3187)  Acc@5: 98.4375 (99.7664)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.07470 (0.273)  Acc@1: 98.4375 (90.3802)  Acc@5: 100.0000 (99.7952)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 0.2949 (0.274)  Acc@1: 87.5000 (90.3720)  Acc@5: 100.0000 (99.8000)LR: 1.150e-02
train_acc 90.372000
Valid: 27 [   0/390]  Loss: 0.6175 (0.618)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)
Valid: 27 [  50/390]  Loss: 0.6907 (0.496)  Acc@1: 81.2500 (84.4975)  Acc@5: 98.4375 (99.2034)
Valid: 27 [ 100/390]  Loss: 0.4384 (0.507)  Acc@1: 84.3750 (84.3595)  Acc@5: 98.4375 (99.0718)
Valid: 27 [ 150/390]  Loss: 0.4570 (0.500)  Acc@1: 89.0625 (84.5302)  Acc@5: 98.4375 (99.1618)
Valid: 27 [ 200/390]  Loss: 0.7448 (0.501)  Acc@1: 79.6875 (84.4139)  Acc@5: 100.0000 (99.2071)
Valid: 27 [ 250/390]  Loss: 0.4255 (0.499)  Acc@1: 89.0625 (84.4995)  Acc@5: 100.0000 (99.2281)
Valid: 27 [ 300/390]  Loss: 0.4657 (0.500)  Acc@1: 84.3750 (84.4632)  Acc@5: 100.0000 (99.2525)
Valid: 27 [ 350/390]  Loss: 0.3162 (0.502)  Acc@1: 90.6250 (84.5264)  Acc@5: 100.0000 (99.2566)
Valid: 27 [ 390/390]  Loss: 0.4056 (0.501)  Acc@1: 90.0000 (84.5680)  Acc@5: 100.0000 (99.2640)
valid_acc 84.568000
epoch = 27   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3951, 0.6049],
        [0.4148, 0.5852],
        [0.3190, 0.6810],
        [0.3128, 0.6872],
        [0.3939, 0.6061],
        [0.2619, 0.7381],
        [0.3635, 0.6365],
        [0.3987, 0.6013],
        [0.3923, 0.6077],
        [0.4898, 0.5102],
        [0.3579, 0.6421],
        [0.4929, 0.5071],
        [0.5778, 0.4222],
        [0.4829, 0.5171]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6431, 0.3569],
        [0.4666, 0.5334],
        [0.6263, 0.3737],
        [0.6044, 0.3956],
        [0.3525, 0.6475],
        [0.6227, 0.3773],
        [0.4455, 0.5545],
        [0.4231, 0.5769],
        [0.4336, 0.5664],
        [0.6320, 0.3680],
        [0.6006, 0.3994],
        [0.4027, 0.5973],
        [0.4072, 0.5928],
        [0.3948, 0.6052]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.3068 (0.307)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (100.0000)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.1715 (0.241)  Acc@1: 92.1875 (91.6973)  Acc@5: 100.0000 (99.9081)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.1999 (0.241)  Acc@1: 89.0625 (91.6770)  Acc@5: 100.0000 (99.8917)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.5056 (0.246)  Acc@1: 76.5625 (91.2666)  Acc@5: 100.0000 (99.8551)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 0.3614 (0.244)  Acc@1: 90.6250 (91.4335)  Acc@5: 100.0000 (99.8756)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.2757 (0.249)  Acc@1: 87.5000 (91.2351)  Acc@5: 100.0000 (99.8257)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 0.3649 (0.257)  Acc@1: 92.1875 (90.9832)  Acc@5: 96.8750 (99.8131)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.3344 (0.260)  Acc@1: 89.0625 (90.8565)  Acc@5: 100.0000 (99.8264)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.1632 (0.262)  Acc@1: 97.5000 (90.8400)  Acc@5: 100.0000 (99.8200)LR: 1.075e-02
train_acc 90.840000
Valid: 28 [   0/390]  Loss: 0.5116 (0.512)  Acc@1: 89.0625 (89.0625)  Acc@5: 96.8750 (96.8750)
Valid: 28 [  50/390]  Loss: 0.4787 (0.480)  Acc@1: 82.8125 (84.4363)  Acc@5: 100.0000 (99.4792)
Valid: 28 [ 100/390]  Loss: 0.4466 (0.494)  Acc@1: 85.9375 (84.4833)  Acc@5: 100.0000 (99.2574)
Valid: 28 [ 150/390]  Loss: 0.6971 (0.484)  Acc@1: 79.6875 (84.6440)  Acc@5: 96.8750 (99.3274)
Valid: 28 [ 200/390]  Loss: 0.6382 (0.486)  Acc@1: 81.2500 (84.5305)  Acc@5: 96.8750 (99.3470)
Valid: 28 [ 250/390]  Loss: 0.4495 (0.484)  Acc@1: 87.5000 (84.5369)  Acc@5: 100.0000 (99.3152)
Valid: 28 [ 300/390]  Loss: 0.5277 (0.486)  Acc@1: 84.3750 (84.4892)  Acc@5: 100.0000 (99.3096)
Valid: 28 [ 350/390]  Loss: 0.4244 (0.484)  Acc@1: 92.1875 (84.5531)  Acc@5: 100.0000 (99.3278)
Valid: 28 [ 390/390]  Loss: 0.2657 (0.481)  Acc@1: 87.5000 (84.6320)  Acc@5: 100.0000 (99.3160)
valid_acc 84.632000
epoch = 28   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('sep_conv_3x3', 1), ('dil_conv_5x5', 0), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3904, 0.6096],
        [0.4105, 0.5895],
        [0.3132, 0.6868],
        [0.3107, 0.6893],
        [0.3878, 0.6122],
        [0.2468, 0.7532],
        [0.3657, 0.6343],
        [0.3995, 0.6005],
        [0.3977, 0.6023],
        [0.4936, 0.5064],
        [0.3457, 0.6543],
        [0.4945, 0.5055],
        [0.5804, 0.4196],
        [0.4779, 0.5221]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6413, 0.3587],
        [0.4557, 0.5443],
        [0.6256, 0.3744],
        [0.6023, 0.3977],
        [0.3463, 0.6537],
        [0.6224, 0.3776],
        [0.4415, 0.5585],
        [0.4199, 0.5801],
        [0.4280, 0.5720],
        [0.6320, 0.3680],
        [0.5994, 0.4006],
        [0.3995, 0.6005],
        [0.4045, 0.5955],
        [0.3868, 0.6132]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.2497 (0.250)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.2157 (0.233)  Acc@1: 92.1875 (92.1262)  Acc@5: 100.0000 (99.9081)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.1035 (0.231)  Acc@1: 98.4375 (92.1875)  Acc@5: 100.0000 (99.8453)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.1165 (0.231)  Acc@1: 95.3125 (92.0426)  Acc@5: 100.0000 (99.8758)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.2669 (0.236)  Acc@1: 87.5000 (91.9854)  Acc@5: 100.0000 (99.8912)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.1566 (0.235)  Acc@1: 95.3125 (91.9385)  Acc@5: 100.0000 (99.8942)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.3518 (0.232)  Acc@1: 89.0625 (92.0006)  Acc@5: 96.8750 (99.8910)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.1947 (0.230)  Acc@1: 93.7500 (92.0540)  Acc@5: 100.0000 (99.8843)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 0.2024 (0.232)  Acc@1: 92.5000 (91.9560)  Acc@5: 100.0000 (99.8920)LR: 1.002e-02
train_acc 91.956000
Valid: 29 [   0/390]  Loss: 0.2988 (0.299)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 29 [  50/390]  Loss: 0.3084 (0.483)  Acc@1: 89.0625 (85.2635)  Acc@5: 100.0000 (99.2953)
Valid: 29 [ 100/390]  Loss: 0.5422 (0.467)  Acc@1: 79.6875 (85.3806)  Acc@5: 98.4375 (99.4585)
Valid: 29 [ 150/390]  Loss: 0.5180 (0.481)  Acc@1: 85.9375 (85.2856)  Acc@5: 98.4375 (99.4205)
Valid: 29 [ 200/390]  Loss: 0.6683 (0.491)  Acc@1: 84.3750 (85.2379)  Acc@5: 98.4375 (99.3781)
Valid: 29 [ 250/390]  Loss: 0.4095 (0.492)  Acc@1: 85.9375 (85.2963)  Acc@5: 100.0000 (99.3588)
Valid: 29 [ 300/390]  Loss: 0.7976 (0.493)  Acc@1: 82.8125 (85.3769)  Acc@5: 93.7500 (99.3304)
Valid: 29 [ 350/390]  Loss: 0.6543 (0.499)  Acc@1: 84.3750 (85.1674)  Acc@5: 98.4375 (99.3011)
Valid: 29 [ 390/390]  Loss: 0.4564 (0.504)  Acc@1: 90.0000 (85.0120)  Acc@5: 95.0000 (99.2680)
valid_acc 85.012000
epoch = 29   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3887, 0.6113],
        [0.4064, 0.5936],
        [0.3073, 0.6927],
        [0.3087, 0.6913],
        [0.3844, 0.6156],
        [0.2341, 0.7659],
        [0.3692, 0.6308],
        [0.4033, 0.5967],
        [0.4066, 0.5934],
        [0.4991, 0.5009],
        [0.3342, 0.6658],
        [0.5009, 0.4991],
        [0.5819, 0.4181],
        [0.4755, 0.5245]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6359, 0.3641],
        [0.4483, 0.5517],
        [0.6214, 0.3786],
        [0.6019, 0.3981],
        [0.3437, 0.6563],
        [0.6173, 0.3827],
        [0.4340, 0.5660],
        [0.4223, 0.5777],
        [0.4281, 0.5719],
        [0.6281, 0.3719],
        [0.5996, 0.4004],
        [0.3986, 0.6014],
        [0.4025, 0.5975],
        [0.3820, 0.6180]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.2768 (0.277)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.2848 (0.234)  Acc@1: 89.0625 (91.9730)  Acc@5: 100.0000 (99.9081)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.2231 (0.231)  Acc@1: 90.6250 (92.0483)  Acc@5: 100.0000 (99.8762)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.1961 (0.227)  Acc@1: 93.7500 (92.2289)  Acc@5: 100.0000 (99.8862)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.2023 (0.226)  Acc@1: 95.3125 (92.3197)  Acc@5: 100.0000 (99.8756)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.2647 (0.233)  Acc@1: 87.5000 (92.0070)  Acc@5: 100.0000 (99.8568)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.2374 (0.238)  Acc@1: 92.1875 (91.8189)  Acc@5: 100.0000 (99.8391)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.1627 (0.237)  Acc@1: 95.3125 (91.7379)  Acc@5: 100.0000 (99.8531)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 0.3713 (0.240)  Acc@1: 87.5000 (91.6320)  Acc@5: 97.5000 (99.8480)LR: 9.292e-03
train_acc 91.632000
Valid: 30 [   0/390]  Loss: 0.6868 (0.687)  Acc@1: 79.6875 (79.6875)  Acc@5: 95.3125 (95.3125)
Valid: 30 [  50/390]  Loss: 0.4028 (0.539)  Acc@1: 89.0625 (83.9767)  Acc@5: 98.4375 (99.0809)
Valid: 30 [ 100/390]  Loss: 0.4262 (0.551)  Acc@1: 82.8125 (83.7407)  Acc@5: 100.0000 (98.9635)
Valid: 30 [ 150/390]  Loss: 0.4198 (0.548)  Acc@1: 89.0625 (83.8266)  Acc@5: 100.0000 (99.0584)
Valid: 30 [ 200/390]  Loss: 0.5102 (0.539)  Acc@1: 85.9375 (83.9008)  Acc@5: 98.4375 (99.0749)
Valid: 30 [ 250/390]  Loss: 0.4155 (0.536)  Acc@1: 85.9375 (83.8085)  Acc@5: 100.0000 (99.0849)
Valid: 30 [ 300/390]  Loss: 0.5464 (0.534)  Acc@1: 84.3750 (83.8559)  Acc@5: 98.4375 (99.1227)
Valid: 30 [ 350/390]  Loss: 0.4725 (0.538)  Acc@1: 85.9375 (83.8275)  Acc@5: 98.4375 (99.1008)
Valid: 30 [ 390/390]  Loss: 0.1995 (0.539)  Acc@1: 95.0000 (83.8000)  Acc@5: 100.0000 (99.1160)
valid_acc 83.800000
epoch = 30   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3866, 0.6134],
        [0.4018, 0.5982],
        [0.3004, 0.6996],
        [0.3108, 0.6892],
        [0.3831, 0.6169],
        [0.2214, 0.7786],
        [0.3715, 0.6285],
        [0.4078, 0.5922],
        [0.4151, 0.5849],
        [0.5009, 0.4991],
        [0.3244, 0.6756],
        [0.5054, 0.4946],
        [0.5786, 0.4214],
        [0.4737, 0.5263]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6330, 0.3670],
        [0.4436, 0.5564],
        [0.6217, 0.3783],
        [0.6002, 0.3998],
        [0.3393, 0.6607],
        [0.6146, 0.3854],
        [0.4316, 0.5684],
        [0.4191, 0.5809],
        [0.4267, 0.5733],
        [0.6280, 0.3720],
        [0.5969, 0.4031],
        [0.3926, 0.6074],
        [0.3986, 0.6014],
        [0.3742, 0.6258]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.3136 (0.314)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.1802 (0.215)  Acc@1: 92.1875 (92.3100)  Acc@5: 100.0000 (99.9081)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.1839 (0.207)  Acc@1: 93.7500 (92.4969)  Acc@5: 100.0000 (99.9072)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.08407 (0.208)  Acc@1: 98.4375 (92.5393)  Acc@5: 100.0000 (99.8551)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.4453 (0.205)  Acc@1: 84.3750 (92.6461)  Acc@5: 100.0000 (99.8756)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.1896 (0.209)  Acc@1: 90.6250 (92.4676)  Acc@5: 100.0000 (99.8879)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 0.3127 (0.217)  Acc@1: 89.0625 (92.1615)  Acc@5: 100.0000 (99.9014)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.1742 (0.220)  Acc@1: 93.7500 (92.0940)  Acc@5: 100.0000 (99.8798)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.3156 (0.220)  Acc@1: 87.5000 (92.1240)  Acc@5: 100.0000 (99.8800)LR: 8.583e-03
train_acc 92.124000
Valid: 31 [   0/390]  Loss: 0.6198 (0.620)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 31 [  50/390]  Loss: 0.2653 (0.558)  Acc@1: 92.1875 (83.7316)  Acc@5: 100.0000 (99.1728)
Valid: 31 [ 100/390]  Loss: 0.6575 (0.517)  Acc@1: 89.0625 (84.7618)  Acc@5: 96.8750 (99.2265)
Valid: 31 [ 150/390]  Loss: 0.3966 (0.515)  Acc@1: 87.5000 (84.6337)  Acc@5: 100.0000 (99.2446)
Valid: 31 [ 200/390]  Loss: 0.2618 (0.503)  Acc@1: 87.5000 (84.8025)  Acc@5: 100.0000 (99.2848)
Valid: 31 [ 250/390]  Loss: 0.3214 (0.494)  Acc@1: 89.0625 (84.9851)  Acc@5: 100.0000 (99.2841)
Valid: 31 [ 300/390]  Loss: 0.7176 (0.493)  Acc@1: 78.1250 (84.9720)  Acc@5: 100.0000 (99.3304)
Valid: 31 [ 350/390]  Loss: 0.5067 (0.491)  Acc@1: 85.9375 (85.0205)  Acc@5: 96.8750 (99.3189)
Valid: 31 [ 390/390]  Loss: 0.2963 (0.491)  Acc@1: 85.0000 (85.0040)  Acc@5: 100.0000 (99.3200)
valid_acc 85.004000
epoch = 31   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3797, 0.6203],
        [0.3993, 0.6007],
        [0.2939, 0.7061],
        [0.3114, 0.6886],
        [0.3798, 0.6202],
        [0.2098, 0.7902],
        [0.3748, 0.6252],
        [0.4096, 0.5904],
        [0.4216, 0.5784],
        [0.5009, 0.4991],
        [0.3142, 0.6858],
        [0.5093, 0.4907],
        [0.5792, 0.4208],
        [0.4764, 0.5236]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6306, 0.3694],
        [0.4366, 0.5634],
        [0.6218, 0.3782],
        [0.6016, 0.3984],
        [0.3329, 0.6671],
        [0.6127, 0.3873],
        [0.4271, 0.5729],
        [0.4150, 0.5850],
        [0.4272, 0.5728],
        [0.6289, 0.3711],
        [0.5995, 0.4005],
        [0.3873, 0.6127],
        [0.3960, 0.6040],
        [0.3727, 0.6273]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.4794 (0.479)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.2123 (0.211)  Acc@1: 93.7500 (92.3407)  Acc@5: 100.0000 (99.9387)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.2686 (0.210)  Acc@1: 90.6250 (92.6516)  Acc@5: 100.0000 (99.9381)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.1185 (0.210)  Acc@1: 96.8750 (92.7152)  Acc@5: 100.0000 (99.9483)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.09427 (0.204)  Acc@1: 98.4375 (92.8871)  Acc@5: 100.0000 (99.9378)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.1344 (0.206)  Acc@1: 95.3125 (92.8411)  Acc@5: 100.0000 (99.9377)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.1617 (0.207)  Acc@1: 93.7500 (92.8312)  Acc@5: 100.0000 (99.9221)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.3089 (0.212)  Acc@1: 89.0625 (92.6282)  Acc@5: 100.0000 (99.9110)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.4696 (0.211)  Acc@1: 92.5000 (92.6560)  Acc@5: 100.0000 (99.9200)LR: 7.891e-03
train_acc 92.656000
Valid: 32 [   0/390]  Loss: 0.4209 (0.421)  Acc@1: 84.3750 (84.3750)  Acc@5: 98.4375 (98.4375)
Valid: 32 [  50/390]  Loss: 0.7391 (0.503)  Acc@1: 76.5625 (84.8652)  Acc@5: 98.4375 (99.3260)
Valid: 32 [ 100/390]  Loss: 0.7303 (0.503)  Acc@1: 79.6875 (84.7153)  Acc@5: 100.0000 (99.3038)
Valid: 32 [ 150/390]  Loss: 0.5514 (0.505)  Acc@1: 84.3750 (84.7682)  Acc@5: 100.0000 (99.2860)
Valid: 32 [ 200/390]  Loss: 0.4350 (0.496)  Acc@1: 84.3750 (84.9658)  Acc@5: 98.4375 (99.2615)
Valid: 32 [ 250/390]  Loss: 0.6642 (0.489)  Acc@1: 79.6875 (85.2403)  Acc@5: 98.4375 (99.3028)
Valid: 32 [ 300/390]  Loss: 0.5228 (0.496)  Acc@1: 85.9375 (85.1069)  Acc@5: 100.0000 (99.2681)
Valid: 32 [ 350/390]  Loss: 0.7371 (0.492)  Acc@1: 79.6875 (85.2119)  Acc@5: 100.0000 (99.2833)
Valid: 32 [ 390/390]  Loss: 1.056 (0.494)  Acc@1: 72.5000 (85.1240)  Acc@5: 97.5000 (99.2880)
valid_acc 85.124000
epoch = 32   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('max_pool_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3729, 0.6271],
        [0.3947, 0.6053],
        [0.2869, 0.7131],
        [0.3112, 0.6888],
        [0.3820, 0.6180],
        [0.2010, 0.7990],
        [0.3756, 0.6244],
        [0.4124, 0.5876],
        [0.4278, 0.5722],
        [0.5074, 0.4926],
        [0.3002, 0.6998],
        [0.5145, 0.4855],
        [0.5816, 0.4184],
        [0.4789, 0.5211]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6285, 0.3715],
        [0.4290, 0.5710],
        [0.6228, 0.3772],
        [0.6044, 0.3956],
        [0.3273, 0.6727],
        [0.6133, 0.3867],
        [0.4236, 0.5764],
        [0.4147, 0.5853],
        [0.4250, 0.5750],
        [0.6297, 0.3703],
        [0.6030, 0.3970],
        [0.3838, 0.6162],
        [0.3918, 0.6082],
        [0.3701, 0.6299]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.09986 (0.0999)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.2505 (0.185)  Acc@1: 87.5000 (93.5355)  Acc@5: 100.0000 (99.9387)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.2153 (0.193)  Acc@1: 92.1875 (93.1931)  Acc@5: 100.0000 (99.9226)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.08091 (0.193)  Acc@1: 98.4375 (93.2016)  Acc@5: 100.0000 (99.9379)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.2825 (0.198)  Acc@1: 87.5000 (92.9960)  Acc@5: 100.0000 (99.9145)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.2150 (0.197)  Acc@1: 93.7500 (93.0403)  Acc@5: 100.0000 (99.9315)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.1834 (0.197)  Acc@1: 93.7500 (92.9713)  Acc@5: 100.0000 (99.9325)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.1271 (0.197)  Acc@1: 93.7500 (93.0021)  Acc@5: 100.0000 (99.9288)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.2438 (0.199)  Acc@1: 92.5000 (92.9040)  Acc@5: 100.0000 (99.9280)LR: 7.219e-03
train_acc 92.904000
Valid: 33 [   0/390]  Loss: 0.4506 (0.451)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 33 [  50/390]  Loss: 0.4726 (0.503)  Acc@1: 90.6250 (85.1409)  Acc@5: 98.4375 (99.1728)
Valid: 33 [ 100/390]  Loss: 0.4324 (0.504)  Acc@1: 89.0625 (85.3496)  Acc@5: 100.0000 (99.2884)
Valid: 33 [ 150/390]  Loss: 0.2817 (0.517)  Acc@1: 92.1875 (84.9752)  Acc@5: 100.0000 (99.2446)
Valid: 33 [ 200/390]  Loss: 0.4234 (0.532)  Acc@1: 81.2500 (84.5305)  Acc@5: 100.0000 (99.2071)
Valid: 33 [ 250/390]  Loss: 0.4491 (0.523)  Acc@1: 87.5000 (84.7547)  Acc@5: 100.0000 (99.2219)
Valid: 33 [ 300/390]  Loss: 0.1487 (0.525)  Acc@1: 95.3125 (84.7176)  Acc@5: 100.0000 (99.2058)
Valid: 33 [ 350/390]  Loss: 0.3409 (0.525)  Acc@1: 90.6250 (84.7311)  Acc@5: 100.0000 (99.2165)
Valid: 33 [ 390/390]  Loss: 0.5647 (0.519)  Acc@1: 85.0000 (84.8160)  Acc@5: 100.0000 (99.2320)
valid_acc 84.816000
epoch = 33   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3713, 0.6287],
        [0.3935, 0.6065],
        [0.2845, 0.7155],
        [0.3125, 0.6875],
        [0.3835, 0.6165],
        [0.1905, 0.8095],
        [0.3762, 0.6238],
        [0.4191, 0.5809],
        [0.4338, 0.5662],
        [0.5099, 0.4901],
        [0.2885, 0.7115],
        [0.5179, 0.4821],
        [0.5834, 0.4166],
        [0.4785, 0.5215]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6213, 0.3787],
        [0.4266, 0.5734],
        [0.6162, 0.3838],
        [0.6016, 0.3984],
        [0.3183, 0.6817],
        [0.6082, 0.3918],
        [0.4210, 0.5790],
        [0.4108, 0.5892],
        [0.4222, 0.5778],
        [0.6241, 0.3759],
        [0.6006, 0.3994],
        [0.3752, 0.6248],
        [0.3888, 0.6112],
        [0.3607, 0.6393]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.1580 (0.158)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.1749 (0.191)  Acc@1: 92.1875 (93.1679)  Acc@5: 100.0000 (99.9081)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.1441 (0.193)  Acc@1: 95.3125 (93.1312)  Acc@5: 100.0000 (99.9381)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.1643 (0.192)  Acc@1: 93.7500 (93.0360)  Acc@5: 100.0000 (99.9379)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.1870 (0.191)  Acc@1: 92.1875 (93.1281)  Acc@5: 100.0000 (99.9378)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.1463 (0.192)  Acc@1: 95.3125 (93.1960)  Acc@5: 100.0000 (99.9253)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.1723 (0.190)  Acc@1: 90.6250 (93.2361)  Acc@5: 100.0000 (99.9169)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.1406 (0.191)  Acc@1: 95.3125 (93.2425)  Acc@5: 100.0000 (99.9154)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 0.08370 (0.193)  Acc@1: 100.0000 (93.1640)  Acc@5: 100.0000 (99.8960)LR: 6.570e-03
train_acc 93.164000
Valid: 34 [   0/390]  Loss: 0.5942 (0.594)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 34 [  50/390]  Loss: 0.7072 (0.561)  Acc@1: 79.6875 (84.1605)  Acc@5: 98.4375 (99.2647)
Valid: 34 [ 100/390]  Loss: 0.5129 (0.554)  Acc@1: 90.6250 (84.5142)  Acc@5: 98.4375 (99.2574)
Valid: 34 [ 150/390]  Loss: 0.2619 (0.527)  Acc@1: 89.0625 (84.8510)  Acc@5: 100.0000 (99.3171)
Valid: 34 [ 200/390]  Loss: 0.3904 (0.525)  Acc@1: 89.0625 (85.0047)  Acc@5: 100.0000 (99.3081)
Valid: 34 [ 250/390]  Loss: 0.2542 (0.531)  Acc@1: 93.7500 (84.9166)  Acc@5: 100.0000 (99.2654)
Valid: 34 [ 300/390]  Loss: 0.5456 (0.538)  Acc@1: 81.2500 (84.6294)  Acc@5: 98.4375 (99.2058)
Valid: 34 [ 350/390]  Loss: 0.4131 (0.532)  Acc@1: 82.8125 (84.7934)  Acc@5: 100.0000 (99.2210)
Valid: 34 [ 390/390]  Loss: 0.3038 (0.530)  Acc@1: 85.0000 (84.8120)  Acc@5: 100.0000 (99.1920)
valid_acc 84.812000
epoch = 34   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3654, 0.6346],
        [0.3883, 0.6117],
        [0.2779, 0.7221],
        [0.3091, 0.6909],
        [0.3819, 0.6181],
        [0.1789, 0.8211],
        [0.3836, 0.6164],
        [0.4204, 0.5796],
        [0.4385, 0.5615],
        [0.5100, 0.4900],
        [0.2760, 0.7240],
        [0.5187, 0.4813],
        [0.5890, 0.4110],
        [0.4737, 0.5263]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6185, 0.3815],
        [0.4215, 0.5785],
        [0.6161, 0.3839],
        [0.5998, 0.4002],
        [0.3130, 0.6870],
        [0.6080, 0.3920],
        [0.4146, 0.5854],
        [0.4113, 0.5887],
        [0.4169, 0.5831],
        [0.6245, 0.3755],
        [0.5996, 0.4004],
        [0.3657, 0.6343],
        [0.3820, 0.6180],
        [0.3580, 0.6420]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.1348 (0.135)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.2202 (0.165)  Acc@1: 93.7500 (94.1789)  Acc@5: 100.0000 (99.9081)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.1687 (0.173)  Acc@1: 95.3125 (93.9202)  Acc@5: 100.0000 (99.9226)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.2373 (0.180)  Acc@1: 92.1875 (93.5948)  Acc@5: 100.0000 (99.9379)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.2105 (0.177)  Acc@1: 95.3125 (93.7500)  Acc@5: 100.0000 (99.9300)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.09356 (0.180)  Acc@1: 95.3125 (93.6504)  Acc@5: 100.0000 (99.9315)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.05526 (0.182)  Acc@1: 100.0000 (93.5631)  Acc@5: 100.0000 (99.9325)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.2045 (0.182)  Acc@1: 93.7500 (93.5541)  Acc@5: 100.0000 (99.9377)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.1515 (0.184)  Acc@1: 92.5000 (93.4760)  Acc@5: 100.0000 (99.9280)LR: 5.947e-03
train_acc 93.476000
Valid: 35 [   0/390]  Loss: 0.3202 (0.320)  Acc@1: 89.0625 (89.0625)  Acc@5: 98.4375 (98.4375)
Valid: 35 [  50/390]  Loss: 0.3869 (0.479)  Acc@1: 89.0625 (86.1213)  Acc@5: 100.0000 (99.2034)
Valid: 35 [ 100/390]  Loss: 0.3870 (0.478)  Acc@1: 84.3750 (85.9375)  Acc@5: 100.0000 (99.3348)
Valid: 35 [ 150/390]  Loss: 0.3109 (0.481)  Acc@1: 87.5000 (85.9168)  Acc@5: 100.0000 (99.2964)
Valid: 35 [ 200/390]  Loss: 0.4811 (0.480)  Acc@1: 89.0625 (85.9064)  Acc@5: 98.4375 (99.2926)
Valid: 35 [ 250/390]  Loss: 0.3963 (0.486)  Acc@1: 85.9375 (85.6885)  Acc@5: 100.0000 (99.2841)
Valid: 35 [ 300/390]  Loss: 0.4572 (0.480)  Acc@1: 87.5000 (85.8337)  Acc@5: 100.0000 (99.2836)
Valid: 35 [ 350/390]  Loss: 0.3406 (0.476)  Acc@1: 89.0625 (85.8574)  Acc@5: 100.0000 (99.3056)
Valid: 35 [ 390/390]  Loss: 0.3522 (0.476)  Acc@1: 87.5000 (85.9320)  Acc@5: 100.0000 (99.3240)
valid_acc 85.932000
epoch = 35   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3595, 0.6405],
        [0.3873, 0.6127],
        [0.2706, 0.7294],
        [0.3076, 0.6924],
        [0.3887, 0.6113],
        [0.1704, 0.8296],
        [0.3858, 0.6142],
        [0.4224, 0.5776],
        [0.4431, 0.5569],
        [0.5114, 0.4886],
        [0.2627, 0.7373],
        [0.5246, 0.4754],
        [0.5914, 0.4086],
        [0.4684, 0.5316]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6068, 0.3932],
        [0.4130, 0.5870],
        [0.6060, 0.3940],
        [0.5960, 0.4040],
        [0.3106, 0.6894],
        [0.5980, 0.4020],
        [0.4093, 0.5907],
        [0.4105, 0.5895],
        [0.4131, 0.5869],
        [0.6152, 0.3848],
        [0.5960, 0.4040],
        [0.3628, 0.6372],
        [0.3777, 0.6223],
        [0.3528, 0.6472]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.06040 (0.0604)  Acc@1: 100.0000 (100.0000)  Acc@5: 100.0000 (100.0000)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.1919 (0.158)  Acc@1: 96.8750 (94.5772)  Acc@5: 100.0000 (99.8775)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.1189 (0.162)  Acc@1: 93.7500 (94.4307)  Acc@5: 100.0000 (99.9072)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.2147 (0.167)  Acc@1: 90.6250 (94.1743)  Acc@5: 100.0000 (99.9276)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.1347 (0.168)  Acc@1: 95.3125 (94.0920)  Acc@5: 100.0000 (99.9456)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.1155 (0.172)  Acc@1: 95.3125 (93.8185)  Acc@5: 100.0000 (99.9502)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.1204 (0.173)  Acc@1: 96.8750 (93.8538)  Acc@5: 100.0000 (99.9429)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.3090 (0.173)  Acc@1: 89.0625 (93.8969)  Acc@5: 100.0000 (99.9421)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.1839 (0.173)  Acc@1: 95.0000 (93.9480)  Acc@5: 100.0000 (99.9240)LR: 5.351e-03
train_acc 93.948000
Valid: 36 [   0/390]  Loss: 0.3433 (0.343)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 36 [  50/390]  Loss: 0.5812 (0.456)  Acc@1: 78.1250 (86.5196)  Acc@5: 100.0000 (99.5404)
Valid: 36 [ 100/390]  Loss: 0.4201 (0.469)  Acc@1: 87.5000 (86.5563)  Acc@5: 100.0000 (99.3193)
Valid: 36 [ 150/390]  Loss: 0.6082 (0.466)  Acc@1: 84.3750 (86.5273)  Acc@5: 100.0000 (99.3584)
Valid: 36 [ 200/390]  Loss: 0.6484 (0.463)  Acc@1: 84.3750 (86.6060)  Acc@5: 98.4375 (99.3470)
Valid: 36 [ 250/390]  Loss: 0.3638 (0.460)  Acc@1: 90.6250 (86.7592)  Acc@5: 100.0000 (99.3650)
Valid: 36 [ 300/390]  Loss: 0.2851 (0.463)  Acc@1: 90.6250 (86.7058)  Acc@5: 100.0000 (99.3719)
Valid: 36 [ 350/390]  Loss: 0.4604 (0.465)  Acc@1: 84.3750 (86.6631)  Acc@5: 100.0000 (99.3323)
Valid: 36 [ 390/390]  Loss: 0.1363 (0.465)  Acc@1: 97.5000 (86.6760)  Acc@5: 100.0000 (99.3080)
valid_acc 86.676000
epoch = 36   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3540, 0.6460],
        [0.3833, 0.6167],
        [0.2645, 0.7355],
        [0.3087, 0.6913],
        [0.3883, 0.6117],
        [0.1601, 0.8399],
        [0.3918, 0.6082],
        [0.4256, 0.5744],
        [0.4504, 0.5496],
        [0.5150, 0.4850],
        [0.2506, 0.7494],
        [0.5243, 0.4757],
        [0.5932, 0.4068],
        [0.4722, 0.5278]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.6020, 0.3980],
        [0.4091, 0.5909],
        [0.6039, 0.3961],
        [0.5947, 0.4053],
        [0.3039, 0.6961],
        [0.5976, 0.4024],
        [0.4059, 0.5941],
        [0.4109, 0.5891],
        [0.4152, 0.5848],
        [0.6159, 0.3841],
        [0.5937, 0.4063],
        [0.3594, 0.6406],
        [0.3735, 0.6265],
        [0.3504, 0.6496]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.1183 (0.118)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.1554 (0.141)  Acc@1: 93.7500 (94.9142)  Acc@5: 100.0000 (99.9694)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.2349 (0.164)  Acc@1: 93.7500 (94.0285)  Acc@5: 100.0000 (99.9226)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.2872 (0.164)  Acc@1: 92.1875 (94.1018)  Acc@5: 98.4375 (99.9276)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.1248 (0.166)  Acc@1: 95.3125 (94.0065)  Acc@5: 100.0000 (99.9145)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.1441 (0.171)  Acc@1: 95.3125 (93.9119)  Acc@5: 100.0000 (99.8942)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.08628 (0.168)  Acc@1: 96.8750 (93.9940)  Acc@5: 100.0000 (99.9066)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.1124 (0.167)  Acc@1: 96.8750 (94.0394)  Acc@5: 100.0000 (99.9154)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.1784 (0.168)  Acc@1: 95.0000 (94.0120)  Acc@5: 100.0000 (99.9160)LR: 4.785e-03
train_acc 94.012000
Valid: 37 [   0/390]  Loss: 0.6514 (0.651)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 37 [  50/390]  Loss: 0.7991 (0.483)  Acc@1: 85.9375 (86.2439)  Acc@5: 98.4375 (99.3260)
Valid: 37 [ 100/390]  Loss: 0.7386 (0.506)  Acc@1: 79.6875 (85.9220)  Acc@5: 98.4375 (99.2265)
Valid: 37 [ 150/390]  Loss: 0.4141 (0.503)  Acc@1: 82.8125 (85.9789)  Acc@5: 100.0000 (99.1825)
Valid: 37 [ 200/390]  Loss: 0.7715 (0.491)  Acc@1: 81.2500 (86.2873)  Acc@5: 96.8750 (99.2071)
Valid: 37 [ 250/390]  Loss: 0.5101 (0.482)  Acc@1: 81.2500 (86.4542)  Acc@5: 100.0000 (99.2841)
Valid: 37 [ 300/390]  Loss: 0.3403 (0.487)  Acc@1: 90.6250 (86.2334)  Acc@5: 100.0000 (99.3148)
Valid: 37 [ 350/390]  Loss: 0.3381 (0.485)  Acc@1: 92.1875 (86.2447)  Acc@5: 100.0000 (99.3323)
Valid: 37 [ 390/390]  Loss: 0.4564 (0.483)  Acc@1: 87.5000 (86.3760)  Acc@5: 100.0000 (99.3440)
valid_acc 86.376000
epoch = 37   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3462, 0.6538],
        [0.3782, 0.6218],
        [0.2557, 0.7443],
        [0.3110, 0.6890],
        [0.3857, 0.6143],
        [0.1526, 0.8474],
        [0.3959, 0.6041],
        [0.4281, 0.5719],
        [0.4535, 0.5465],
        [0.5135, 0.4865],
        [0.2413, 0.7587],
        [0.5263, 0.4737],
        [0.5916, 0.4084],
        [0.4675, 0.5325]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5961, 0.4039],
        [0.4030, 0.5970],
        [0.6017, 0.3983],
        [0.5889, 0.4111],
        [0.2999, 0.7001],
        [0.5939, 0.4061],
        [0.3969, 0.6031],
        [0.4069, 0.5931],
        [0.4113, 0.5887],
        [0.6143, 0.3857],
        [0.5894, 0.4106],
        [0.3587, 0.6413],
        [0.3694, 0.6306],
        [0.3531, 0.6469]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.1945 (0.195)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.1123 (0.159)  Acc@1: 98.4375 (94.2096)  Acc@5: 100.0000 (99.9694)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.1383 (0.156)  Acc@1: 93.7500 (94.3224)  Acc@5: 100.0000 (99.9536)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.1868 (0.150)  Acc@1: 93.7500 (94.6399)  Acc@5: 100.0000 (99.9483)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.1589 (0.151)  Acc@1: 95.3125 (94.6284)  Acc@5: 100.0000 (99.9534)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.1652 (0.150)  Acc@1: 95.3125 (94.5717)  Acc@5: 100.0000 (99.9564)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.2081 (0.151)  Acc@1: 93.7500 (94.5702)  Acc@5: 100.0000 (99.9533)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.1732 (0.151)  Acc@1: 93.7500 (94.5424)  Acc@5: 100.0000 (99.9599)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.1364 (0.151)  Acc@1: 95.0000 (94.5560)  Acc@5: 100.0000 (99.9640)LR: 4.252e-03
train_acc 94.556000
Valid: 38 [   0/390]  Loss: 0.4296 (0.430)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 38 [  50/390]  Loss: 0.3952 (0.503)  Acc@1: 87.5000 (86.3664)  Acc@5: 98.4375 (99.1728)
Valid: 38 [ 100/390]  Loss: 0.5262 (0.499)  Acc@1: 85.9375 (86.0767)  Acc@5: 100.0000 (99.2574)
Valid: 38 [ 150/390]  Loss: 0.5005 (0.491)  Acc@1: 85.9375 (86.1962)  Acc@5: 96.8750 (99.2343)
Valid: 38 [ 200/390]  Loss: 0.3666 (0.485)  Acc@1: 90.6250 (86.4583)  Acc@5: 100.0000 (99.2537)
Valid: 38 [ 250/390]  Loss: 0.5523 (0.483)  Acc@1: 82.8125 (86.3048)  Acc@5: 98.4375 (99.2405)
Valid: 38 [ 300/390]  Loss: 0.6008 (0.481)  Acc@1: 81.2500 (86.3580)  Acc@5: 100.0000 (99.2525)
Valid: 38 [ 350/390]  Loss: 0.4319 (0.486)  Acc@1: 92.1875 (86.3292)  Acc@5: 100.0000 (99.2967)
Valid: 38 [ 390/390]  Loss: 0.6427 (0.483)  Acc@1: 82.5000 (86.3560)  Acc@5: 97.5000 (99.3280)
valid_acc 86.356000
epoch = 38   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3411, 0.6589],
        [0.3707, 0.6293],
        [0.2531, 0.7469],
        [0.3144, 0.6856],
        [0.3896, 0.6104],
        [0.1448, 0.8552],
        [0.4006, 0.5994],
        [0.4338, 0.5662],
        [0.4606, 0.5394],
        [0.5109, 0.4891],
        [0.2343, 0.7657],
        [0.5262, 0.4738],
        [0.5914, 0.4086],
        [0.4668, 0.5332]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5904, 0.4096],
        [0.3965, 0.6035],
        [0.5990, 0.4010],
        [0.5858, 0.4142],
        [0.2954, 0.7046],
        [0.5915, 0.4085],
        [0.3912, 0.6088],
        [0.4010, 0.5990],
        [0.4069, 0.5931],
        [0.6119, 0.3881],
        [0.5855, 0.4145],
        [0.3545, 0.6455],
        [0.3652, 0.6348],
        [0.3476, 0.6524]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.2779 (0.278)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.08131 (0.143)  Acc@1: 95.3125 (94.5466)  Acc@5: 100.0000 (99.9387)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.1480 (0.139)  Acc@1: 95.3125 (94.9567)  Acc@5: 100.0000 (99.9381)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.1594 (0.141)  Acc@1: 90.6250 (94.9296)  Acc@5: 100.0000 (99.9586)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.2840 (0.140)  Acc@1: 90.6250 (95.0326)  Acc@5: 100.0000 (99.9689)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.2515 (0.142)  Acc@1: 90.6250 (95.0012)  Acc@5: 100.0000 (99.9626)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.2129 (0.144)  Acc@1: 90.6250 (94.9491)  Acc@5: 100.0000 (99.9637)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.1799 (0.145)  Acc@1: 93.7500 (94.9297)  Acc@5: 100.0000 (99.9599)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.2099 (0.146)  Acc@1: 90.0000 (94.8520)  Acc@5: 100.0000 (99.9600)LR: 3.754e-03
train_acc 94.852000
Valid: 39 [   0/390]  Loss: 0.5191 (0.519)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 39 [  50/390]  Loss: 0.7375 (0.475)  Acc@1: 78.1250 (86.4583)  Acc@5: 98.4375 (99.4179)
Valid: 39 [ 100/390]  Loss: 0.3999 (0.515)  Acc@1: 84.3750 (85.7364)  Acc@5: 98.4375 (99.4121)
Valid: 39 [ 150/390]  Loss: 0.6225 (0.505)  Acc@1: 84.3750 (86.0099)  Acc@5: 98.4375 (99.4102)
Valid: 39 [ 200/390]  Loss: 0.3949 (0.498)  Acc@1: 90.6250 (86.1940)  Acc@5: 98.4375 (99.3859)
Valid: 39 [ 250/390]  Loss: 0.1933 (0.509)  Acc@1: 93.7500 (86.0682)  Acc@5: 100.0000 (99.3339)
Valid: 39 [ 300/390]  Loss: 0.6229 (0.511)  Acc@1: 78.1250 (85.9946)  Acc@5: 100.0000 (99.3252)
Valid: 39 [ 350/390]  Loss: 0.5469 (0.514)  Acc@1: 85.9375 (85.9330)  Acc@5: 100.0000 (99.3056)
Valid: 39 [ 390/390]  Loss: 0.5353 (0.514)  Acc@1: 85.0000 (85.8920)  Acc@5: 100.0000 (99.3000)
valid_acc 85.892000
epoch = 39   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3346, 0.6654],
        [0.3702, 0.6298],
        [0.2464, 0.7536],
        [0.3145, 0.6855],
        [0.3841, 0.6159],
        [0.1371, 0.8629],
        [0.3996, 0.6004],
        [0.4380, 0.5620],
        [0.4685, 0.5315],
        [0.5136, 0.4864],
        [0.2244, 0.7756],
        [0.5235, 0.4765],
        [0.5893, 0.4107],
        [0.4660, 0.5340]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5853, 0.4147],
        [0.3873, 0.6127],
        [0.5958, 0.4042],
        [0.5835, 0.4165],
        [0.2906, 0.7094],
        [0.5889, 0.4111],
        [0.3871, 0.6129],
        [0.3933, 0.6067],
        [0.4031, 0.5969],
        [0.6103, 0.3897],
        [0.5831, 0.4169],
        [0.3533, 0.6467],
        [0.3607, 0.6393],
        [0.3428, 0.6572]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.1039 (0.104)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.1762 (0.139)  Acc@1: 95.3125 (95.1287)  Acc@5: 100.0000 (99.9081)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.07971 (0.132)  Acc@1: 96.8750 (95.3434)  Acc@5: 100.0000 (99.9381)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.1358 (0.129)  Acc@1: 93.7500 (95.4470)  Acc@5: 100.0000 (99.9483)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.1616 (0.132)  Acc@1: 95.3125 (95.4524)  Acc@5: 100.0000 (99.9534)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.1910 (0.131)  Acc@1: 95.3125 (95.4495)  Acc@5: 100.0000 (99.9564)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.1109 (0.131)  Acc@1: 95.3125 (95.4215)  Acc@5: 100.0000 (99.9585)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.2018 (0.133)  Acc@1: 93.7500 (95.3926)  Acc@5: 100.0000 (99.9510)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.2032 (0.135)  Acc@1: 90.0000 (95.3120)  Acc@5: 100.0000 (99.9560)LR: 3.292e-03
train_acc 95.312000
Valid: 40 [   0/390]  Loss: 0.5887 (0.589)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)
Valid: 40 [  50/390]  Loss: 0.2952 (0.516)  Acc@1: 87.5000 (85.7230)  Acc@5: 100.0000 (99.3566)
Valid: 40 [ 100/390]  Loss: 0.5374 (0.521)  Acc@1: 89.0625 (85.9530)  Acc@5: 100.0000 (99.3038)
Valid: 40 [ 150/390]  Loss: 0.3361 (0.511)  Acc@1: 87.5000 (86.1651)  Acc@5: 100.0000 (99.3481)
Valid: 40 [ 200/390]  Loss: 0.7874 (0.499)  Acc@1: 79.6875 (86.3029)  Acc@5: 98.4375 (99.3470)
Valid: 40 [ 250/390]  Loss: 0.6189 (0.500)  Acc@1: 79.6875 (86.3421)  Acc@5: 100.0000 (99.3526)
Valid: 40 [ 300/390]  Loss: 0.5543 (0.499)  Acc@1: 84.3750 (86.2801)  Acc@5: 98.4375 (99.3823)
Valid: 40 [ 350/390]  Loss: 0.5207 (0.502)  Acc@1: 85.9375 (86.2135)  Acc@5: 100.0000 (99.3723)
Valid: 40 [ 390/390]  Loss: 0.4435 (0.498)  Acc@1: 90.0000 (86.3640)  Acc@5: 100.0000 (99.3640)
valid_acc 86.364000
epoch = 40   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3301, 0.6699],
        [0.3682, 0.6318],
        [0.2408, 0.7592],
        [0.3147, 0.6853],
        [0.3830, 0.6170],
        [0.1262, 0.8738],
        [0.4019, 0.5981],
        [0.4413, 0.5587],
        [0.4789, 0.5211],
        [0.5155, 0.4845],
        [0.2134, 0.7866],
        [0.5276, 0.4724],
        [0.5854, 0.4146],
        [0.4681, 0.5319]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5748, 0.4252],
        [0.3812, 0.6188],
        [0.5901, 0.4099],
        [0.5806, 0.4194],
        [0.2828, 0.7172],
        [0.5840, 0.4160],
        [0.3824, 0.6176],
        [0.3829, 0.6171],
        [0.3969, 0.6031],
        [0.6064, 0.3936],
        [0.5809, 0.4191],
        [0.3490, 0.6510],
        [0.3570, 0.6430],
        [0.3401, 0.6599]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.08259 (0.0826)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.2002 (0.124)  Acc@1: 93.7500 (95.9252)  Acc@5: 100.0000 (99.9694)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.1051 (0.127)  Acc@1: 93.7500 (95.7611)  Acc@5: 100.0000 (99.9845)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.09688 (0.128)  Acc@1: 95.3125 (95.5815)  Acc@5: 100.0000 (99.9690)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.08037 (0.129)  Acc@1: 98.4375 (95.4913)  Acc@5: 100.0000 (99.9611)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.09458 (0.128)  Acc@1: 96.8750 (95.5179)  Acc@5: 100.0000 (99.9626)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.06578 (0.129)  Acc@1: 98.4375 (95.4267)  Acc@5: 100.0000 (99.9585)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.1536 (0.130)  Acc@1: 90.6250 (95.4060)  Acc@5: 100.0000 (99.9466)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.08864 (0.130)  Acc@1: 97.5000 (95.4200)  Acc@5: 100.0000 (99.9520)LR: 2.868e-03
train_acc 95.420000
Valid: 41 [   0/390]  Loss: 0.4968 (0.497)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)
Valid: 41 [  50/390]  Loss: 0.5172 (0.487)  Acc@1: 84.3750 (86.3664)  Acc@5: 100.0000 (99.3566)
Valid: 41 [ 100/390]  Loss: 0.5385 (0.490)  Acc@1: 81.2500 (86.1386)  Acc@5: 100.0000 (99.3812)
Valid: 41 [ 150/390]  Loss: 0.4490 (0.490)  Acc@1: 89.0625 (86.1548)  Acc@5: 100.0000 (99.3377)
Valid: 41 [ 200/390]  Loss: 0.2135 (0.486)  Acc@1: 92.1875 (86.1474)  Acc@5: 100.0000 (99.4014)
Valid: 41 [ 250/390]  Loss: 0.5193 (0.487)  Acc@1: 85.9375 (86.1803)  Acc@5: 100.0000 (99.3899)
Valid: 41 [ 300/390]  Loss: 0.2656 (0.494)  Acc@1: 92.1875 (86.2490)  Acc@5: 100.0000 (99.3719)
Valid: 41 [ 350/390]  Loss: 0.7778 (0.495)  Acc@1: 81.2500 (86.2803)  Acc@5: 100.0000 (99.3768)
Valid: 41 [ 390/390]  Loss: 0.3539 (0.491)  Acc@1: 95.0000 (86.4080)  Acc@5: 100.0000 (99.3880)
valid_acc 86.408000
epoch = 41   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3240, 0.6760],
        [0.3688, 0.6312],
        [0.2325, 0.7675],
        [0.3178, 0.6822],
        [0.3820, 0.6180],
        [0.1191, 0.8809],
        [0.4069, 0.5931],
        [0.4478, 0.5522],
        [0.4902, 0.5098],
        [0.5191, 0.4809],
        [0.2050, 0.7950],
        [0.5291, 0.4709],
        [0.5807, 0.4193],
        [0.4690, 0.5310]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5707, 0.4293],
        [0.3763, 0.6237],
        [0.5877, 0.4123],
        [0.5778, 0.4222],
        [0.2778, 0.7222],
        [0.5834, 0.4166],
        [0.3800, 0.6200],
        [0.3807, 0.6193],
        [0.3950, 0.6050],
        [0.6053, 0.3947],
        [0.5778, 0.4222],
        [0.3507, 0.6493],
        [0.3617, 0.6383],
        [0.3380, 0.6620]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.09377 (0.0938)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.1013 (0.120)  Acc@1: 95.3125 (95.9559)  Acc@5: 100.0000 (99.9694)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.07937 (0.125)  Acc@1: 98.4375 (95.7611)  Acc@5: 100.0000 (99.9381)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.2376 (0.123)  Acc@1: 92.1875 (95.8092)  Acc@5: 98.4375 (99.9483)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.1029 (0.123)  Acc@1: 96.8750 (95.7245)  Acc@5: 100.0000 (99.9534)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.1510 (0.122)  Acc@1: 93.7500 (95.7358)  Acc@5: 100.0000 (99.9502)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.2435 (0.124)  Acc@1: 87.5000 (95.6551)  Acc@5: 100.0000 (99.9585)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.1189 (0.124)  Acc@1: 93.7500 (95.6597)  Acc@5: 100.0000 (99.9599)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.2954 (0.125)  Acc@1: 92.5000 (95.6000)  Acc@5: 100.0000 (99.9640)LR: 2.484e-03
train_acc 95.600000
Valid: 42 [   0/390]  Loss: 0.6386 (0.639)  Acc@1: 82.8125 (82.8125)  Acc@5: 100.0000 (100.0000)
Valid: 42 [  50/390]  Loss: 0.1204 (0.506)  Acc@1: 93.7500 (86.2745)  Acc@5: 100.0000 (99.4792)
Valid: 42 [ 100/390]  Loss: 0.6818 (0.498)  Acc@1: 85.9375 (86.1850)  Acc@5: 98.4375 (99.3348)
Valid: 42 [ 150/390]  Loss: 0.4974 (0.506)  Acc@1: 90.6250 (86.1134)  Acc@5: 98.4375 (99.3171)
Valid: 42 [ 200/390]  Loss: 0.3830 (0.509)  Acc@1: 89.0625 (86.2174)  Acc@5: 100.0000 (99.3237)
Valid: 42 [ 250/390]  Loss: 0.4198 (0.505)  Acc@1: 81.2500 (86.2176)  Acc@5: 100.0000 (99.3775)
Valid: 42 [ 300/390]  Loss: 0.3848 (0.499)  Acc@1: 89.0625 (86.2801)  Acc@5: 100.0000 (99.3926)
Valid: 42 [ 350/390]  Loss: 0.5616 (0.495)  Acc@1: 89.0625 (86.4761)  Acc@5: 100.0000 (99.3634)
Valid: 42 [ 390/390]  Loss: 0.2103 (0.495)  Acc@1: 92.5000 (86.4280)  Acc@5: 100.0000 (99.3520)
valid_acc 86.428000
epoch = 42   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3202, 0.6798],
        [0.3637, 0.6363],
        [0.2269, 0.7731],
        [0.3205, 0.6795],
        [0.3838, 0.6162],
        [0.1135, 0.8865],
        [0.4139, 0.5861],
        [0.4484, 0.5516],
        [0.4950, 0.5050],
        [0.5185, 0.4815],
        [0.1966, 0.8034],
        [0.5317, 0.4683],
        [0.5796, 0.4204],
        [0.4722, 0.5278]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5658, 0.4342],
        [0.3749, 0.6251],
        [0.5866, 0.4134],
        [0.5758, 0.4242],
        [0.2732, 0.7268],
        [0.5827, 0.4173],
        [0.3755, 0.6245],
        [0.3803, 0.6197],
        [0.3951, 0.6049],
        [0.6046, 0.3954],
        [0.5782, 0.4218],
        [0.3455, 0.6545],
        [0.3614, 0.6386],
        [0.3386, 0.6614]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.1266 (0.127)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.08718 (0.126)  Acc@1: 96.8750 (95.5882)  Acc@5: 100.0000 (99.9694)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.2085 (0.115)  Acc@1: 96.8750 (96.1479)  Acc@5: 98.4375 (99.9691)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.1513 (0.114)  Acc@1: 95.3125 (96.1610)  Acc@5: 100.0000 (99.9793)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.05361 (0.113)  Acc@1: 98.4375 (96.1598)  Acc@5: 100.0000 (99.9845)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.09856 (0.113)  Acc@1: 96.8750 (96.1840)  Acc@5: 100.0000 (99.9875)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.1570 (0.117)  Acc@1: 92.1875 (95.9406)  Acc@5: 100.0000 (99.9896)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.07635 (0.119)  Acc@1: 96.8750 (95.8333)  Acc@5: 100.0000 (99.9911)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.1088 (0.118)  Acc@1: 95.0000 (95.8720)  Acc@5: 100.0000 (99.9920)LR: 2.142e-03
train_acc 95.872000
Valid: 43 [   0/390]  Loss: 0.3478 (0.348)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 43 [  50/390]  Loss: 0.5425 (0.510)  Acc@1: 85.9375 (86.1826)  Acc@5: 98.4375 (99.2647)
Valid: 43 [ 100/390]  Loss: 0.2832 (0.529)  Acc@1: 89.0625 (85.9839)  Acc@5: 100.0000 (99.2110)
Valid: 43 [ 150/390]  Loss: 0.4356 (0.510)  Acc@1: 90.6250 (86.4031)  Acc@5: 98.4375 (99.2860)
Valid: 43 [ 200/390]  Loss: 0.4561 (0.517)  Acc@1: 87.5000 (86.0541)  Acc@5: 100.0000 (99.3626)
Valid: 43 [ 250/390]  Loss: 0.4712 (0.515)  Acc@1: 87.5000 (86.1180)  Acc@5: 98.4375 (99.3899)
Valid: 43 [ 300/390]  Loss: 0.2692 (0.518)  Acc@1: 89.0625 (86.0361)  Acc@5: 98.4375 (99.4030)
Valid: 43 [ 350/390]  Loss: 0.4602 (0.517)  Acc@1: 87.5000 (86.1111)  Acc@5: 100.0000 (99.3812)
Valid: 43 [ 390/390]  Loss: 0.5737 (0.515)  Acc@1: 85.0000 (86.1200)  Acc@5: 100.0000 (99.4000)
valid_acc 86.120000
epoch = 43   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3181, 0.6819],
        [0.3598, 0.6402],
        [0.2196, 0.7804],
        [0.3213, 0.6787],
        [0.3814, 0.6186],
        [0.1086, 0.8914],
        [0.4158, 0.5842],
        [0.4513, 0.5487],
        [0.4995, 0.5005],
        [0.5224, 0.4776],
        [0.1909, 0.8091],
        [0.5370, 0.4630],
        [0.5768, 0.4232],
        [0.4776, 0.5224]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5590, 0.4410],
        [0.3713, 0.6287],
        [0.5858, 0.4142],
        [0.5738, 0.4262],
        [0.2721, 0.7279],
        [0.5826, 0.4174],
        [0.3703, 0.6297],
        [0.3795, 0.6205],
        [0.3920, 0.6080],
        [0.6041, 0.3959],
        [0.5754, 0.4246],
        [0.3442, 0.6558],
        [0.3605, 0.6395],
        [0.3416, 0.6584]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.08984 (0.0898)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.1454 (0.0976)  Acc@1: 93.7500 (96.4767)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.1185 (0.103)  Acc@1: 95.3125 (96.4264)  Acc@5: 100.0000 (99.9536)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.04189 (0.112)  Acc@1: 100.0000 (96.1093)  Acc@5: 100.0000 (99.9690)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.09833 (0.110)  Acc@1: 93.7500 (96.1521)  Acc@5: 100.0000 (99.9767)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.06570 (0.110)  Acc@1: 96.8750 (96.0969)  Acc@5: 100.0000 (99.9751)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.1015 (0.113)  Acc@1: 92.1875 (95.9199)  Acc@5: 100.0000 (99.9689)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.06734 (0.115)  Acc@1: 98.4375 (95.9001)  Acc@5: 100.0000 (99.9644)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.07316 (0.114)  Acc@1: 97.5000 (95.9600)  Acc@5: 100.0000 (99.9640)LR: 1.843e-03
train_acc 95.960000
Valid: 44 [   0/390]  Loss: 0.5440 (0.544)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 44 [  50/390]  Loss: 0.3463 (0.446)  Acc@1: 85.9375 (87.8370)  Acc@5: 100.0000 (99.4485)
Valid: 44 [ 100/390]  Loss: 0.3712 (0.451)  Acc@1: 87.5000 (87.5000)  Acc@5: 100.0000 (99.4121)
Valid: 44 [ 150/390]  Loss: 0.1910 (0.471)  Acc@1: 92.1875 (87.1068)  Acc@5: 100.0000 (99.4205)
Valid: 44 [ 200/390]  Loss: 0.8985 (0.469)  Acc@1: 81.2500 (87.1813)  Acc@5: 98.4375 (99.4248)
Valid: 44 [ 250/390]  Loss: 0.3350 (0.468)  Acc@1: 92.1875 (87.3257)  Acc@5: 100.0000 (99.4460)
Valid: 44 [ 300/390]  Loss: 0.4603 (0.469)  Acc@1: 87.5000 (87.2197)  Acc@5: 100.0000 (99.4498)
Valid: 44 [ 350/390]  Loss: 0.3584 (0.475)  Acc@1: 92.1875 (87.0548)  Acc@5: 100.0000 (99.4614)
Valid: 44 [ 390/390]  Loss: 0.5919 (0.479)  Acc@1: 90.0000 (86.9200)  Acc@5: 100.0000 (99.4600)
valid_acc 86.920000
epoch = 44   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3156, 0.6844],
        [0.3538, 0.6462],
        [0.2168, 0.7832],
        [0.3240, 0.6760],
        [0.3823, 0.6177],
        [0.1038, 0.8962],
        [0.4212, 0.5788],
        [0.4561, 0.5439],
        [0.5019, 0.4981],
        [0.5277, 0.4723],
        [0.1832, 0.8168],
        [0.5421, 0.4579],
        [0.5778, 0.4222],
        [0.4790, 0.5210]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5532, 0.4468],
        [0.3613, 0.6387],
        [0.5872, 0.4128],
        [0.5728, 0.4272],
        [0.2664, 0.7336],
        [0.5823, 0.4177],
        [0.3662, 0.6338],
        [0.3770, 0.6230],
        [0.3869, 0.6131],
        [0.6053, 0.3947],
        [0.5742, 0.4258],
        [0.3408, 0.6592],
        [0.3618, 0.6382],
        [0.3373, 0.6627]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.07901 (0.0790)  Acc@1: 98.4375 (98.4375)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.1404 (0.115)  Acc@1: 93.7500 (95.6189)  Acc@5: 100.0000 (99.9387)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.08486 (0.109)  Acc@1: 96.8750 (95.9623)  Acc@5: 100.0000 (99.9381)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.1356 (0.111)  Acc@1: 93.7500 (95.8713)  Acc@5: 100.0000 (99.9586)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.1205 (0.112)  Acc@1: 96.8750 (95.9266)  Acc@5: 100.0000 (99.9611)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.08358 (0.112)  Acc@1: 95.3125 (95.9910)  Acc@5: 100.0000 (99.9626)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.08593 (0.112)  Acc@1: 98.4375 (95.9666)  Acc@5: 100.0000 (99.9637)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.06117 (0.111)  Acc@1: 96.8750 (96.0158)  Acc@5: 100.0000 (99.9688)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.04069 (0.110)  Acc@1: 100.0000 (96.0760)  Acc@5: 100.0000 (99.9720)LR: 1.587e-03
train_acc 96.076000
Valid: 45 [   0/390]  Loss: 0.5595 (0.559)  Acc@1: 84.3750 (84.3750)  Acc@5: 100.0000 (100.0000)
Valid: 45 [  50/390]  Loss: 0.8355 (0.522)  Acc@1: 79.6875 (86.3971)  Acc@5: 100.0000 (99.2953)
Valid: 45 [ 100/390]  Loss: 0.6207 (0.521)  Acc@1: 84.3750 (86.3397)  Acc@5: 98.4375 (99.3502)
Valid: 45 [ 150/390]  Loss: 0.5760 (0.516)  Acc@1: 87.5000 (86.5791)  Acc@5: 98.4375 (99.3274)
Valid: 45 [ 200/390]  Loss: 0.5142 (0.521)  Acc@1: 84.3750 (86.2329)  Acc@5: 98.4375 (99.3470)
Valid: 45 [ 250/390]  Loss: 1.031 (0.520)  Acc@1: 73.4375 (86.2674)  Acc@5: 96.8750 (99.3028)
Valid: 45 [ 300/390]  Loss: 0.5712 (0.513)  Acc@1: 82.8125 (86.3268)  Acc@5: 98.4375 (99.2940)
Valid: 45 [ 350/390]  Loss: 0.4782 (0.511)  Acc@1: 85.9375 (86.3649)  Acc@5: 100.0000 (99.2922)
Valid: 45 [ 390/390]  Loss: 0.5325 (0.510)  Acc@1: 92.5000 (86.3720)  Acc@5: 97.5000 (99.3080)
valid_acc 86.372000
epoch = 45   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 4), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3137, 0.6863],
        [0.3484, 0.6516],
        [0.2125, 0.7875],
        [0.3225, 0.6775],
        [0.3768, 0.6232],
        [0.0993, 0.9007],
        [0.4226, 0.5774],
        [0.4577, 0.5423],
        [0.5074, 0.4926],
        [0.5289, 0.4711],
        [0.1751, 0.8249],
        [0.5448, 0.4552],
        [0.5758, 0.4242],
        [0.4845, 0.5155]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5496, 0.4504],
        [0.3568, 0.6432],
        [0.5857, 0.4143],
        [0.5699, 0.4301],
        [0.2618, 0.7382],
        [0.5810, 0.4190],
        [0.3668, 0.6332],
        [0.3749, 0.6251],
        [0.3833, 0.6167],
        [0.6052, 0.3948],
        [0.5724, 0.4276],
        [0.3382, 0.6618],
        [0.3607, 0.6393],
        [0.3379, 0.6621]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.2437 (0.244)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.06480 (0.109)  Acc@1: 96.8750 (95.9252)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.05230 (0.112)  Acc@1: 100.0000 (95.9468)  Acc@5: 100.0000 (99.9691)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.1154 (0.110)  Acc@1: 95.3125 (96.1403)  Acc@5: 100.0000 (99.9793)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.1052 (0.110)  Acc@1: 98.4375 (96.1287)  Acc@5: 100.0000 (99.9767)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.05197 (0.109)  Acc@1: 100.0000 (96.1529)  Acc@5: 100.0000 (99.9751)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.07133 (0.106)  Acc@1: 98.4375 (96.2365)  Acc@5: 100.0000 (99.9792)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.06760 (0.106)  Acc@1: 98.4375 (96.2918)  Acc@5: 100.0000 (99.9822)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.05280 (0.106)  Acc@1: 97.5000 (96.2880)  Acc@5: 100.0000 (99.9840)LR: 1.377e-03
train_acc 96.288000
Valid: 46 [   0/390]  Loss: 0.2240 (0.224)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)
Valid: 46 [  50/390]  Loss: 0.3417 (0.439)  Acc@1: 90.6250 (87.7145)  Acc@5: 100.0000 (99.5098)
Valid: 46 [ 100/390]  Loss: 0.5372 (0.468)  Acc@1: 87.5000 (87.1442)  Acc@5: 98.4375 (99.3657)
Valid: 46 [ 150/390]  Loss: 0.4882 (0.474)  Acc@1: 89.0625 (87.3241)  Acc@5: 100.0000 (99.3688)
Valid: 46 [ 200/390]  Loss: 0.5517 (0.480)  Acc@1: 85.9375 (87.1424)  Acc@5: 100.0000 (99.3703)
Valid: 46 [ 250/390]  Loss: 0.3811 (0.486)  Acc@1: 92.1875 (87.0456)  Acc@5: 100.0000 (99.3837)
Valid: 46 [ 300/390]  Loss: 0.5944 (0.486)  Acc@1: 84.3750 (87.0069)  Acc@5: 100.0000 (99.4238)
Valid: 46 [ 350/390]  Loss: 0.4729 (0.494)  Acc@1: 89.0625 (86.8412)  Acc@5: 98.4375 (99.4124)
Valid: 46 [ 390/390]  Loss: 0.3994 (0.488)  Acc@1: 90.0000 (86.9440)  Acc@5: 100.0000 (99.4440)
valid_acc 86.944000
epoch = 46   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3099, 0.6901],
        [0.3420, 0.6580],
        [0.2088, 0.7912],
        [0.3187, 0.6813],
        [0.3778, 0.6222],
        [0.0951, 0.9049],
        [0.4237, 0.5763],
        [0.4613, 0.5387],
        [0.5088, 0.4912],
        [0.5258, 0.4742],
        [0.1674, 0.8326],
        [0.5507, 0.4493],
        [0.5743, 0.4257],
        [0.4865, 0.5135]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5472, 0.4528],
        [0.3537, 0.6463],
        [0.5867, 0.4133],
        [0.5668, 0.4332],
        [0.2563, 0.7437],
        [0.5813, 0.4187],
        [0.3606, 0.6394],
        [0.3718, 0.6282],
        [0.3827, 0.6173],
        [0.6070, 0.3930],
        [0.5692, 0.4308],
        [0.3328, 0.6672],
        [0.3607, 0.6393],
        [0.3375, 0.6625]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.07212 (0.0721)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.2118 (0.104)  Acc@1: 93.7500 (96.4154)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.1068 (0.103)  Acc@1: 96.8750 (96.5811)  Acc@5: 100.0000 (99.9845)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.1144 (0.103)  Acc@1: 98.4375 (96.6163)  Acc@5: 100.0000 (99.9897)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.08933 (0.0997)  Acc@1: 95.3125 (96.6107)  Acc@5: 100.0000 (99.9922)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.2875 (0.101)  Acc@1: 92.1875 (96.5700)  Acc@5: 100.0000 (99.9875)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.06835 (0.102)  Acc@1: 98.4375 (96.4961)  Acc@5: 100.0000 (99.9792)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.08023 (0.102)  Acc@1: 95.3125 (96.5011)  Acc@5: 100.0000 (99.9822)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.02529 (0.103)  Acc@1: 100.0000 (96.4960)  Acc@5: 100.0000 (99.9800)LR: 1.213e-03
train_acc 96.496000
Valid: 47 [   0/390]  Loss: 0.6557 (0.656)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)
Valid: 47 [  50/390]  Loss: 0.6237 (0.488)  Acc@1: 85.9375 (86.6115)  Acc@5: 100.0000 (99.3566)
Valid: 47 [ 100/390]  Loss: 0.2309 (0.516)  Acc@1: 92.1875 (86.2005)  Acc@5: 100.0000 (99.2574)
Valid: 47 [ 150/390]  Loss: 0.4268 (0.508)  Acc@1: 85.9375 (86.3307)  Acc@5: 100.0000 (99.3377)
Valid: 47 [ 200/390]  Loss: 0.4733 (0.501)  Acc@1: 82.8125 (86.4117)  Acc@5: 100.0000 (99.4014)
Valid: 47 [ 250/390]  Loss: 0.3386 (0.497)  Acc@1: 89.0625 (86.5040)  Acc@5: 100.0000 (99.4335)
Valid: 47 [ 300/390]  Loss: 0.4693 (0.500)  Acc@1: 85.9375 (86.4774)  Acc@5: 98.4375 (99.3875)
Valid: 47 [ 350/390]  Loss: 0.3605 (0.508)  Acc@1: 89.0625 (86.3070)  Acc@5: 100.0000 (99.3768)
Valid: 47 [ 390/390]  Loss: 0.4589 (0.509)  Acc@1: 85.0000 (86.2200)  Acc@5: 100.0000 (99.3640)
valid_acc 86.220000
epoch = 47   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3078, 0.6922],
        [0.3380, 0.6620],
        [0.2046, 0.7954],
        [0.3170, 0.6830],
        [0.3752, 0.6248],
        [0.0921, 0.9079],
        [0.4255, 0.5745],
        [0.4608, 0.5392],
        [0.5109, 0.4891],
        [0.5263, 0.4737],
        [0.1624, 0.8376],
        [0.5559, 0.4441],
        [0.5752, 0.4248],
        [0.4917, 0.5083]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5454, 0.4546],
        [0.3506, 0.6494],
        [0.5876, 0.4124],
        [0.5680, 0.4320],
        [0.2527, 0.7473],
        [0.5823, 0.4177],
        [0.3557, 0.6443],
        [0.3675, 0.6325],
        [0.3824, 0.6176],
        [0.6097, 0.3903],
        [0.5696, 0.4304],
        [0.3295, 0.6705],
        [0.3576, 0.6424],
        [0.3376, 0.6624]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.1737 (0.174)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.05623 (0.0940)  Acc@1: 98.4375 (96.6605)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.1980 (0.102)  Acc@1: 93.7500 (96.3800)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.04211 (0.101)  Acc@1: 98.4375 (96.3059)  Acc@5: 100.0000 (100.0000)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.08487 (0.0983)  Acc@1: 96.8750 (96.4708)  Acc@5: 100.0000 (99.9922)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.2425 (0.0999)  Acc@1: 93.7500 (96.4579)  Acc@5: 100.0000 (99.9938)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.05133 (0.100)  Acc@1: 96.8750 (96.4701)  Acc@5: 100.0000 (99.9948)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.1447 (0.0995)  Acc@1: 93.7500 (96.4655)  Acc@5: 100.0000 (99.9955)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.1809 (0.0997)  Acc@1: 92.5000 (96.4400)  Acc@5: 100.0000 (99.9920)LR: 1.095e-03
train_acc 96.440000
Valid: 48 [   0/390]  Loss: 0.4002 (0.400)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)
Valid: 48 [  50/390]  Loss: 0.6508 (0.538)  Acc@1: 85.9375 (86.2745)  Acc@5: 100.0000 (99.3566)
Valid: 48 [ 100/390]  Loss: 0.4738 (0.543)  Acc@1: 84.3750 (86.3243)  Acc@5: 100.0000 (99.3657)
Valid: 48 [ 150/390]  Loss: 0.4991 (0.536)  Acc@1: 90.6250 (86.2479)  Acc@5: 100.0000 (99.3895)
Valid: 48 [ 200/390]  Loss: 0.4670 (0.523)  Acc@1: 85.9375 (86.5827)  Acc@5: 98.4375 (99.3392)
Valid: 48 [ 250/390]  Loss: 0.9364 (0.521)  Acc@1: 84.3750 (86.5538)  Acc@5: 100.0000 (99.3837)
Valid: 48 [ 300/390]  Loss: 0.7318 (0.514)  Acc@1: 85.9375 (86.6487)  Acc@5: 100.0000 (99.3719)
Valid: 48 [ 350/390]  Loss: 0.5268 (0.513)  Acc@1: 85.9375 (86.6898)  Acc@5: 98.4375 (99.3412)
Valid: 48 [ 390/390]  Loss: 0.4333 (0.511)  Acc@1: 90.0000 (86.6440)  Acc@5: 100.0000 (99.3520)
valid_acc 86.644000
epoch = 48   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3025, 0.6975],
        [0.3366, 0.6634],
        [0.2019, 0.7981],
        [0.3164, 0.6836],
        [0.3781, 0.6219],
        [0.0874, 0.9126],
        [0.4268, 0.5732],
        [0.4662, 0.5338],
        [0.5188, 0.4812],
        [0.5255, 0.4745],
        [0.1583, 0.8417],
        [0.5621, 0.4379],
        [0.5770, 0.4230],
        [0.4915, 0.5085]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5412, 0.4588],
        [0.3457, 0.6543],
        [0.5867, 0.4133],
        [0.5672, 0.4328],
        [0.2477, 0.7523],
        [0.5815, 0.4185],
        [0.3552, 0.6448],
        [0.3638, 0.6362],
        [0.3823, 0.6177],
        [0.6081, 0.3919],
        [0.5682, 0.4318],
        [0.3242, 0.6758],
        [0.3575, 0.6425],
        [0.3369, 0.6631]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.09260 (0.0926)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.04396 (0.0913)  Acc@1: 100.0000 (97.1201)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.07980 (0.0974)  Acc@1: 96.8750 (96.9678)  Acc@5: 100.0000 (99.9691)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.06423 (0.0996)  Acc@1: 98.4375 (96.7405)  Acc@5: 100.0000 (99.9586)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.08091 (0.0989)  Acc@1: 95.3125 (96.6884)  Acc@5: 100.0000 (99.9689)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.04942 (0.0997)  Acc@1: 98.4375 (96.5886)  Acc@5: 100.0000 (99.9751)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.04849 (0.101)  Acc@1: 96.8750 (96.5635)  Acc@5: 100.0000 (99.9792)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.1558 (0.101)  Acc@1: 96.8750 (96.5411)  Acc@5: 100.0000 (99.9822)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.1841 (0.103)  Acc@1: 90.0000 (96.4720)  Acc@5: 100.0000 (99.9800)LR: 1.024e-03
train_acc 96.472000
Valid: 49 [   0/390]  Loss: 0.5200 (0.520)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)
Valid: 49 [  50/390]  Loss: 0.4766 (0.490)  Acc@1: 87.5000 (86.9485)  Acc@5: 100.0000 (99.2034)
Valid: 49 [ 100/390]  Loss: 0.8308 (0.525)  Acc@1: 79.6875 (86.4944)  Acc@5: 98.4375 (99.1491)
Valid: 49 [ 150/390]  Loss: 0.5200 (0.503)  Acc@1: 84.3750 (86.8688)  Acc@5: 100.0000 (99.3171)
Valid: 49 [ 200/390]  Loss: 0.4242 (0.507)  Acc@1: 89.0625 (86.9248)  Acc@5: 98.4375 (99.3315)
Valid: 49 [ 250/390]  Loss: 0.6199 (0.513)  Acc@1: 81.2500 (86.6098)  Acc@5: 100.0000 (99.3464)
Valid: 49 [ 300/390]  Loss: 0.4257 (0.513)  Acc@1: 84.3750 (86.4774)  Acc@5: 100.0000 (99.3252)
Valid: 49 [ 350/390]  Loss: 0.8681 (0.514)  Acc@1: 81.2500 (86.4806)  Acc@5: 100.0000 (99.3234)
Valid: 49 [ 390/390]  Loss: 0.4306 (0.518)  Acc@1: 90.0000 (86.4560)  Acc@5: 100.0000 (99.3040)
valid_acc 86.456000
epoch = 49   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('skip_connect', 0), ('sep_conv_3x3', 1), ('skip_connect', 1), ('dil_conv_3x3', 3)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 4)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2969, 0.7031],
        [0.3330, 0.6670],
        [0.1995, 0.8005],
        [0.3133, 0.6867],
        [0.3756, 0.6244],
        [0.0840, 0.9160],
        [0.4293, 0.5707],
        [0.4657, 0.5343],
        [0.5205, 0.4795],
        [0.5266, 0.4734],
        [0.1526, 0.8474],
        [0.5641, 0.4359],
        [0.5731, 0.4269],
        [0.4953, 0.5047]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5371, 0.4629],
        [0.3433, 0.6567],
        [0.5854, 0.4146],
        [0.5674, 0.4326],
        [0.2451, 0.7549],
        [0.5807, 0.4193],
        [0.3546, 0.6454],
        [0.3606, 0.6394],
        [0.3800, 0.6200],
        [0.6067, 0.3933],
        [0.5695, 0.4305],
        [0.3243, 0.6757],
        [0.3592, 0.6408],
        [0.3368, 0.6632]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
